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1.  INTRODUCTION 


The  goal  of  this  project  was  to  develop  a  field  prototype  medical  imaging  system 
utilizing  3-D  volumetric  ultrasound.  This  goal  was  realized  with  development  of  the 
MUSTPAC™  (Medical  UltraSound,  Three-dimensional,  Portable,  with  Advanced 
Communications).  MUSTPAC™  is  an  ultrasound  telemedicine  system  that  allows  a 
patient  to  be  scanned  at  any  convenient  location  by  an  operator  with  limited  training  and 
no  ultrasound  diagnostic  skills,  then  have  those  scans  analyzed  by  a  diagnostic  expert  at  a 
center  of  excellence  at  any  location.  This  technology  may  have  widespread  application 
in  both  military  healthcare  and  civilian  rural  and  emergency  healthcare,  resulting  in  both 
improved  quality  of  care  and  reduced  costs. 

2.  BODY 

2.1.  BACKGROUND 

Inexpensive,  portable  diagnostic  imaging  systems  can  play  a  key  role  in  decreasing 
battlefield  fatalities  and  reducing  the  cost  of  military  health  care.  Ultrasound  imaging  is 
a  particularly  promising  modality  because  it  does  not  use  ionizing  radiation  (unlike  X- 
rays),  does  not  require  large,  heavy  equipment  (unlike  magnetic  resonance  imaging),  and 
has  been  shown  through  long  use  to  be  safe  and  effective  when  used  by  highly  trained 
practitioners. 

However,  in  current  practice,  ultrasound  is  basically  an  online  two-dimensional  (2-D) 
scanning  procedure  that  produces  a  sequence  of  images  under  interactive  hands-on 
control  by  the  diagnostician.  Each  image  represents  a  slice  through  the  body  at  the 
corresponding  ultrasound  probe  position.  These  images  typically  are  difficult  to 
interpret,  requiring  a  trained  ultrasonographer  with  years  of  experience  to  make  more 
than  the  simplest  diagnoses.  This  need  for  an  expert  interpreter  makes  it  attractive  to  use 
ultrasound  in  a  telemedicine  setting,  sending  images  from  the  patient’s  location  to  a 
skilled  diagnostician  somewhere  else.  However,  with  conventional  2-D  ultrasound, 
considerable  skill  is  required  even  to  position  the  sensor  probe,  since  this  must  be  done 
interactively  as  diagnosis  progresses.  Thus  it  is  problematic  to  use  conventional  2-D 
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ultrasound  in  a  telemedicine  setting,  due  to  the  need  for  a  highly  skilled  operator  to  scan 
the  patient. 

Three-dimensional  (3-D)  ultrasound  imaging  offers  the  potential  to  overcome  these 
difficulties,  thus  providing  a  diagnostically  valuable,  low-cost,  real-time  imaging 
modality  suitable  for  operation  and  use  under  emergency  conditions  by  non-specialists. 
Because  3-D  volumes  show  more  context  than  2-D  slices,  it  becomes  easier  for  users  to 
understand  spatial  relationships  and  detect  abnormal  conditions.  Positioning  the  sensor 
so  as  to  acquire  useful  images  is  also  easier  with  3-D,  because  volumetric  data  can 
readily  be  rotated  and  realigned  to  good  viewing  positions,  largely  independent  of  the 
original  sensor  position.  This  potentially  allows  useful  3-D  ultrasound  data  to  be  taken 
by  an  inexperienced  operator,  then  transmitted  to  and  interpreted  by  a  remote  expert. 

2.2.  CHRONOLOGY  AND  DISCUSSION  OF  ACCOMPLISHMENTS 

In  early  1994,  the  above  considerations  prompted  Battelle  to  submit  to  DARPA,  in 
response  to  solicitation  BAA94-14,  a  proposal  titled  “Real-Time  High  Resolution  3-D 
Ultrasonic  Imaging  for  Physiological  Monitoring”.  This  proposal  laid  out  the  vision  of  a 
three  stage  effort,  roughly  8  years  in  length,  leading  to  the  development  of  an  imaging 
“bed”,  roughly  10,000  square  centimeters  in  size,  containing  an  array  of  high  resolution 
ultrasonic  transducers  and  providing  real-time  3-D  visualization  of  many  physiological 
and  anatomical  structures. 

The  first  stage  of  this  vision,  and  the  focus  of  the  proposal,  was  a  planned  3 -year  project 
to  develop  a  field  prototype  Advanced  IMaging  System  (AIMS).  This  prototype  would 
consist  of  a  lightweight,  portable  ultrasonic  imaging  system  envisioned  as  containing  a  5 
cm  by  5  cm  two-dimensional  transducer  array,  computer  hardware  and  software  for  real¬ 
time  3-D  holographic  image  reconstruction  and  visualization,  and  a  stereo  vision  headset 
for  3-D  image  display.  The  system  was  envisioned  as  being  used  to  rapidly  detect 
foreign  objects  and  bleeding  in  the  body  cavity,  lungs,  or  extremities. 

The  Battelle  proposal  to  develop  an  AIMS  prototype  was  accepted  by  DARPA,  and  the 
project  began  in  September  1994  with  two  major  components: 
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•  Research  and  develop  advanced  sensor  technology,  in  particular,  2-D  transducer 
arrays  utilizing  computational  holographic  focusing  to  acquire  3-D  images  in  real 
time. 

•  Research  and  develop  one  or  more  fully  functional  prototype  systems,  suitable  for 
clinical  and/or  field  use,  to  investigate  and  demonstrate  the  utility  of  3-D  ultrasound 
as  a  medical  imaging  tool  for  use  by  non-specialist  operators. 

In  early  stages  of  the  projects,  it  was  planned  that  these  two  components  would  proceed 
sequentially,  with  the  prototype  system(s)  being  based  on  newly  developed  2-D  array 
transducers  and  thus  appearing  late  in  the  project. 

However,  as  the  project  progressed,  it  became  apparent  that  a  more  productive  strategy 
was  to  pursue  both  components  in  parallel,  with  prototype  systems  being  based  on 
currently  available  1-D  array  transducer  technology.  This  strategy  was  adopted  in  FY95, 
and  research  and  development  continued  along  both  lines  of  research  for  the  next  three 
years. 

In  FY95,  the  primary  activities  and  results  were  as  follows: 

•  Sensor  technology:  Laboratory  research  using  mechanically  scanned  simulations  of 
2-D  transducer  arrays  confirmed  that  high  quality  images,  fully  focused  everywhere 
in  the  3-D  field  of  view,  could  be  obtained  by  using  computational  holographic 
focusing  techniques  in  conjunction  with  large  dense  arrays  (128x128)  of  high- 
frequency  (5  MHz)  transducers.  (See  Appendices  A  and  B.)  However,  the  supporting 
electronics  and  computational  requirements  to  fabricate  and  use  such  large  arrays  for 
real-time  imaging  appeared  beyond  the  reach  of  current  technology.  These 
requirements  could  be  met  with  smaller,  lower  frequency  arrays,  such  as  32x32  at 
1MHz,  and  such  arrays  became  the  focus  for  sensor  investigation  in  FY96. 

•  Prototype  systems:  a  clinically  usable  3-D  ultrasound  system  based  on  “  sequential 
B-scan”  technology  (mechanical  sweep  of  a  conventional  ultrasound  probe)  was 
developed.  This  system  was  displayed  at  the  October  1995  annual  meeting  of  the 
AUSA  (Association  of  the  U.S.  Army)  in  Washington  DC,  where  it  was  favorably 
reviewed  by  many  Army  personnel.  (See  presentation  slides,  Appendix  C)  More 
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importantly,  the  system  was  placed  into  use  in  the  clinic  of  Dr.  Christian  Macedonia 
at  Madigan  Army  Medical  Center  in  Tacoma,  Washington,  for  an  extended 
evaluation. 

In  FY96,  research  and  development  continued  in  both  of  these  areas,  but  with  a  further 
shift  of  emphasis  toward  the  development  of  field-usable  systems.  During  this  period 
(October  1995  through  September  1996),  activities  fell  primarily  in  four  technical  areas: 

•  Evaluation  of  2-D  array  transducers  operating  in  simultaneous  source-receive  mode 
at  an  array  size  and  frequency  consistent  with  then-current  fabrication  technology 
(32x32, 1  MHz),  as  applied  to  abdominal  blood  pooling  using  a  custom  phantom. 

Details  of  the  laboratory  portion  of  this  study  are  contained  in  the  report  provided 
here  as  (Appendix  D):  “Evaluation  of  3-D  Ultrasound  Holographic  Imaging  at  1 
MHz  for  Detection  and  Visualization  of  Abdominal  Blood  Pooling  Using  a 
Mechanically  Scanned  Single  Channel  Transducer  and  Laboratory  Phantoms”. 

Several  medical  consultants  reviewed  this  report  (Dr.  Richard  Satava,  Walter  Reed 
Medical  Center;  Dr.  Christian  Macedonia,  Madigan  Army  Medical  Center;  Dr. 
Stephen  Carter,  University  of  Washington;  and  Dr.  Jonathan  Ophir,  University  of 
Texas). 

The  consensus  opinion  was  that  the  images  would  be  at  best  marginally  useful  for 
diagnosis,  and  that  higher  resolution  (larger  arrays  and  higher  frequencies)  would  be 
required  to  construct  a  useful  system.  Since  larger  arrays  and  higher  frequencies 
were  not  practical  at  that  time  using  simultaneous  source-receive  mode,  this  line  of 
research  was  tabled. 

•  Evaluation  of  computational  holographic  focusing  used  in  conjunction  with  2-D  array 
transducers  operating  in  receive-only  mode  (and  thus  capable  of  being  fabricated  in 
larger  arrays  operating  at  higher  frequency).  This  evaluation  produced  positive 
results,  concluding,  “Three-dimensional  imaging  using  reconstruction  techniques 
developed  at  PNNL  allows  fully-focused  three-dimensional  imaging  with  an 
unlimited  depth  of  field  using  a  2-D  array,  with  or  without  a  lens.  If  a  lens  is  used, 
known  aberrations  that  it  causes  may  be  removed  from  the  images  algorithmically.” 
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Details  of  this  study  are  provided  in  Appendix  E:  “Coherent  Lens-based  Image 
Reconstruction”,  D.  M.  Sheen,  April  3, 1996. 

•  Evaluation  of  the  FY95-produced  “  AIMS  prototype” ,  as  used  in  clinical  studies  by 
Dr. Christian  Macedonia  at  Madigan  Army  Medical  Center. 

Results  from  this  evaluation  were  not  formally  reported,  but  instead  were 
incorporated  directly  into  the  requirements  and  system  design  of  the  MUSTPAC-1. 

•  Development  and  field-testing  of  a  successor  prototype  3-D  ultrasound  telemedicine 
system,  called  the  MUSTPAC-1.  This  system  preserved  the  high  level  architecture  of 
the  earlier  “AIMS  prototype”,  but  replaced  virtually  every  hardware  and  software 
component  of  the  system  with  redesigned  state-of-the-art  elements. 

As  delivered,  the  MUSTPAC-1  consisted  of  an  85-pound  backpack  unit  containing 
everything  necessary  to  operate  in  telemedicine  mode  when  connected  to  any 
standard  network  supporting  TCP/IP.  It  included  the  following  subsystems  and 
capabilities: 

•  Hitachi’s  Model  905  portable,  battery-powered,  2-D  ultrasound  scanner. 

•  Battelle’s  3-D  Paddle  electromechanical  scanner. 

•  TelelnViVo™  (Fraunhofer  CRCG)  volumetric  visualization  software, 
customized  to  meet  this  project’s  requirements. 

•  A  “virtual  ultrasound  probe”,  consisting  of  an  Immersion  Probe™ 

(Immersion  Corporation)  modified  and  interfaced  by  Battelle  and  Fraunhofer 
to  provide  the  look,  feel,  and  effect  of  a  conventional  ultrasound  probe,  when 
computationally  reslicing  volumetric  data. 

•  Silicon  Graphics  Indy™  computer  and  Presenter™  flat-panel  display. 

•  Drag-and-drop  and  graphical  user  interfaces  for  all  functions. 

Delivery  and  field  demonstration  of  the  MUSTPAC-1  was  accomplished  during  July- 
September  1996.  On  July  8,  Battelle  delivered  a  MUSTPAC-1  system  to  the  U.S. 
Army  at  Ft.  Detrick.  Following  compatibility  testing  under  supervision  of  the  Center 
for  Total  Access  (CTA,  Ft.Gordon,  Georgia),  the  MUSTPAC-1  was  shipped  to 
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Landstuhl,  Germany,  for  further  evaluation.  On  August  7,  it  was  deployed  to  the 
212th  Mobile  Army  Surgical  Hospital  in  Tuzla,  Bosnia,  while  a  second  MUSTPAC-1 
remained  at  LRMC  (Landstuhl  Regional  Medical  Center)  to  serve  as  a  receiving 
station.  Additional  secondary  receiving  stations  were  established  and  used  at 
Madigan  Army  Medical  Center  (Tacoma,  WA,  USA),  Fraunhofer  Center  for 
Research  in  Computer  Graphics  (Providence,  RI,  USA),  and  Georgetown  University 
Medical  Center  (Washington,  DC,  USA).  The  MUSTPAC-1  remained  in  Bosnia 
until  it  was  redeployed  to  Georgetown  on  Sept.8. 

This  testing  yielded  the  following  results  (as  summarized  to  DARPA  by  email  from 
Rik  Littlefield  to  Rick  Satava  on  Oct.21, 1996): 

In  early  July,  this  project  delivered  a  prototype  fully 
functional  3-D  ultrasound  telemedicine  system  called  the 
MUSTPAC-1  (Military  Ultrasound,  Three-dimensional  and  Portable, 
with  Advanced  Communication) . 

MUSTPAC-1  provides  the  unique  capability  that  high  quality 
ultrasound  scans  can  be  taken  at  forward  locations  by  an 
operator  with  no  diagnostic  skills,  little  training,  and  no 
online  connection  to  an  expert.  The  scans  are  then  transmitted 
over  any  standard  digital  network  to  a  qualified  diagnostician, 
who  interprets  the  scan  using  a  "virtual  ultrasound  probe”  that 
simulates  a  conventional  real-time  hands-on  examination 
procedure.  The  virtual  probe  and  corresponding  screen  displays 
are  very  natural  to  diagnosticians,  leading  to  rapid  acceptance 
and  productivity  (see  below) . 

In  August  1996,  the  MUSTPAC-1  was  deployed  to  the  212th  MASH  in 
Tuzla,  Boznia.  A  second  MUSTPAC  (minus  some  packaging)  was 
deployed  to  Landstuhl  Regional  Medical  Center  in  Germany  to 
serve  as  a  "buddy”  and  consulting  workstation  for  the  Tuzla 
unit.  During  the  32 -day  deployment,  approximately  72  scans  of 
38  patients  were  taken  at  Tuzla,  with  an  additional  5  scans  of  3 
patients  taken  at  Landstuhl.  Scans  were  exchanged  between 
Tuzla,  Landstuhl,  and  Madigan  Army  Medical  Center  (Tacoma,  WA, 
USA)  using  three  different  telecomm  networks:  the 
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Tuzla/Landstuhl/Tazar  teleradiology  net,  the  Tuzla  TACNET,  and 
an  Inmarsat  satellite  link  (Landstuhl  to  Washington  DC) . 

The  MUSTPAC  system  worked  perfectly. 

Trainup  times  at  Landstuhl  were  observed  to  be  under  5  minutes 
for  diagnosticians  to  go  from  first  contact  with  the  equipment 
to  making  medical  interpretations  from  the  3D  datasets.  In 
Bosnia,  most  of  the  scans  were  taken  by  people  with  no 
ultrasound  experience,  and  less  than  20  minutes  exposure  to  the 
equipment.  Image  quality  in  the  reconstructed  scans  was 
excellent . 

Further  information  on  the  MUSTPAC- 1  and  the  field  test  are  provided  in 
Appendix  F:  “MUSTPAC-1:  3-D  Ultrasound  Telemedicine  System”  (Web  page) 
and  Appendix  G:  "MUSTPAC-1:  3-D  Ultrasound  Telemedicine  Tool  for 
Deployment  Situations  in  Bosnia  and  the  European  Theater" 

As  a  result  of  the  success  of  MUSTPAC-1,  a  change  within  scope  was  negotiated  in  late 
FY96  that  adjusted  the  project’s  priorities  to  reduce  the  level  of  effort  on  2-D  array 
development,  emphasizing  instead  further  development  of  field-usable  systems  based  on 
sequential  B-scan  technology. 

In  FY97,  the  principle  activities  and  accomplishments  under  these  refocused  priorities 
were  as  follows: 

1.  further  demonstration  and  publication  of  the  prototype  MUSTPAC-1  telemedicine 
system,  including  acceptance  of  one  national  award  for  Technology  Innovation; 

2.  patent  application  for  the  MUSTPAC-1  architecture  and  system  concept; 

3.  technical  development  of  a  third-generation  prototype  MUSTPAC-2  system; 

4.  laying  groundwork  for  technology  transfer  of  MUSTPAC-2;  and 

5.  a  small  amount  of  continued  research  on  sensor  technology. 

In  more  detail,  FY-97  activities  and  accomplishments  were  the  following: 

1 .  Further  demonstration  and  publication  of  MUSTPAC- 1 . 
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On  May  31, 1997,  largely  as  a  result  of  the  Germany/Bosnia  field  demonstration, 
MUSTPAC-1  won  the  Discover  Award  for  Technology  Innovation  in  Computer 
Hardware  and  Electronics  (Discover  Magazine,  July  1997). 

Demonstrations  and  discussions  about  MUSTPAC™  were  invited  and  given  — 
mostly  with  funding  from  other  organizations  —  at  the  Association  of  the  United 
States  Army  (AUSA)  Annual  Meeting  (Washington,  DC,  14-16  Oct  96),  the  AUSA 
Telemedicine  Conference  (Tysons  Comer,  VA,  4-6  Mar  97),  the  Military  Medical 
Capabilities  Conference  (Knoxville,  TN,  30  May  97),  Grand  Rounds  at  Walter  Reed 
Army  Medical  Center  and  Georgetown  University  Medical  Center  (Feb  97  and  May 
97),  the  Spanish  Society  of  OB/GYN  meeting  (Spain,  Jun  1997),  the  Society  of 
Minimally  Invasive  Therapeutics  Meeting  (Japan,  Jul  97),  the  Medicine  Meets 
Virtual  Reality  5  (MMVR-5)  conference  (San  Diego,  CA,  Jan  97),  NASA  Johnson 
Space  Center  (Houston,  TX,  Sep  96  and  Jan  97),  the  Tribal  Healthcare  2000 
Conference  (San  Diego,  CA,  15-17  Jul  97),  and  the  U.S.  Army  Medical  Command’s 
Army  Medical  Department  (AMEDD)  Center  and  School  Conference  on  Force 
Structure  and  Requirements  in  Telemedicine  (Jul  1997). 

During  this  period,  the  MUSTPAC-1  received  a  high  level  of  attention  and  support 
from  the  public  news  media.  Technical  articles  about  MUSTPAC™,  written  by 
magazine  staff,  appeared  in  Jane’s  International  Defense  Review  (Feb  97,  pg.15,  see 
Appendix  I)  and  Portable  Design  Magazine  (cover  article,  Jun  97,  see  Appendix  H). 
A  non-technical  article,  associated  with  the  Discover  Award,  appeared  in  Discover 
Magazine  (Jul  1997,  pp.74-75.)  In  addition,  many  non-technical  articles  appeared  in 
AP  newspapers  and  on  broadcast  television,  including  CBS,  CNBC,  the  Discovery 
Channel,  and  the  Disney  Channel. 

2.  Patent  application. 

On  June  25, 1997,  a  patent  application  titled  “Ultrasound  Telemedicine  System  with 
Virtual  Reality”  was  filed  in  support  of  the  U.S.  Department  of  Defense,  covering  the 
system  concept  and  certain  details  of  the  MUSTPAC-1  system.  A  copy  of  this  patent 
application  was  provided  in  the  FY97  annual  report. 

3.  Technical  development  of  MUSTPAC-2. 
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As  of  October  1997,  technical  development  of  the  MUSTPAC-2  was  estimated  as 
being  approximately  70%  complete,  with  the  following  major  actions  accomplished: 

•  Redesigned  the  3-D  Paddle  electromechanical  scanner  to  provide  a  fully  sealed 
operating  mechanism  capable  of  immersion  cleaning  in  standard  disinfectant 
solutions.  This  was  accomplished  through  the  use  of  magnetic  coupling  of 
physical  force  from  the  internal  drive  mechanism,  through  an  impermeable 
aluminum  shell,  to  an  external  probe  carrier. 

•  Replaced  the  Silicon  Graphics  computer,  display,  and  keyboard  with  a  Pentium 
laptop  (Toshiba  Tecra  740CDT)  augmented  with  a  separate  video  capture  card 
(Osprey  100). 

•  Converted  all  software  to  run  under  the  Solaris  x86  operating  system.  (Solaris 
was  an  interim  step,  chosen  to  produce  an  operational  prototype  MUSTPAC-2  on 
the  shortest  possible  schedule.  The  operating  system  for  the  final  version  of 
MUSTPAC™  was  targeted  as  Windows  NT.) 

Although  these  changes  were  conceptually  straightforward,  some  of  them  proved 
surprisingly  difficult  to  accomplish.  Video  capture  was  an  important  example. 

Based  on  discussions  with  vendors  in  1995  and  1996,  MUSTPAC™  development 
staff  believed  that  adequate  video  capture  capability  (640x480  pixels  at  15  fps  or 
better)  would  be  conveniently  available  in  Pentium  laptops,  either  built-in  or  as  a 
CardBus  module.  However,  it  turned  out  that  neither  capability  was  developed  by 
industry  according  to  the  projected  schedule,  and  after  several  false  starts,  the 
MUSTPAC™  project  was  eventually  forced  to  use  a  PCI-bus  video  capture  card. 

This  in  turn  required  adding  an  external  PCI-bus  capability  to  MUSTPAC-2,  with 
attendant  increase  in  size,  weight,  and  complexity  of  packaging.  In  addition,  a 
significant  amount  of  programmer  effort  was  required  to  adapt  vendor-supplied  video 
capture  device  driver  software  to  the  special  needs  of  the  MUSTPAC  project. 

In  retrospect,  it  could  be  seen  that  rapid  development  of  the  high  quality  MUSTPAC- 
1  prototype  was  in  large  part  due  to  the  availability  and  selection  of  a  particularly 
well-suited  computer  platform.  The  Silicon  Graphics  Indy  computer  used  in  the 
MUSTPAC- 1  provided  an  integrated  system,  fully  supported  by  a  single  vendor. 
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encompassing  all  of  the  required  capabilities  in  video  capture,  display,  computational 
speed,  and  communication  software.  It  was  a  noticeable  setback  to  the  project  that 
Silicon  Graphics  chose  to  discontinue  the  Indy,  replacing  it  only  with  systems  that 
were  perceived  as  being  too  large  and  heavy  to  be  viable  for  continued  MUSTPAC™ 
development. 

As  of  October  1997,  however,  many  of  the  difficulties  of  platform  conversion  were 
overcome  and  a  prototype  MUSTPAC-2  system  was  operational.  Further  evaluation 
and  development  were  planned  for  FY98,  leading  to  projected  completion  of  the 
MUSTPAC-2  final  version  in  September  1998. 

4.  Technical  transfer  of  MUSTPAC-2. 

Business  discussions  were  held  with  4  established  ultrasound  companies,  leading  to 
one  agreement-in-principle  for  manufacture  of  MUSTPAC™  systems. 

5.  Development  of  sensor  technology. 

An  improved  method  was  developed  for  computational  holographic  reconstruction  of 
data  obtained  from  a  2-D  array  sensor  using  off-axis  illumination.  This  work  was  a 
small  (3%  of  FY97  project  expenditures)  computational  feasibility  study  designed  to 
complement  other  DARPA-fimded  research  exploring  the  possibility  of  using  a 
receive-only  dense  array  to  achieve  true  real-time  (e.g.  20  fps)  high  resolution  3-D 
acquisition. 

In  summary,  at  the  end  of  FY97,  project  resources  were  focused  almost  completely  on 
development  of  MUSTPAC™,  with  the  goals  of  1)  having  a  Windows  NT  version 
completed  by  September  1998,  and  2)  further  demonstrations  and  evaluations  of  the 
MUSTPAC™  technology. 

During  FY98  and  FY99,  the  MUSTPAC™  technology  was  developed  along  the  general 
lines  planned  in  FY97,  with  refinements  and  redirections  as  determined  by  intermediate 
results  such  as  those  identified  in  the  MUSTPAC™  presentation  at  the  DARPA 
Ultrasound  Workshop  (Feb  1 1-13, 1998,  Lansdowne,  VA)  (see  Appendix  J). 

In  March  1998,  a  contract  modification  and  time  extension  was  negotiated  that 
formalized  three  deliverables  for  the  remainder  of  the  contract.  Quoting  from  email 
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communications  dated  March  20, 1998,  with  Dr.  Wally  Smith,  the  DARPA  program 
manager  for  this  project,  these  deliverables  were  as  follows.  (Specific  progress  toward 
these  deliverables  is  described  in  the  following  section.) 

1.  Mt.  Everest  expedition  field  test. 

Field-test  MUSTPAC™  as  part  of  the  NASA/Yale  Everest  Extreme  Expedition.  One 
MUSTPAC™  unit  will  be  ruggedized  for  cold  and  high  altitude  conditions.  That  unit 
and  one  MUSTPAC™  project  team  member  will  accompany  the  Everest  Extreme 
Expedition  to  Base  Camp,  17600  ft. 

At  Base  Camp  and  in  transit,  the  MUSTPAC™  will  be  tested  for  its  ability  to  collect 
medical  ultrasound  datasets  and  transmit  those  datasets  over  satellite  links  back  to 
consulting  sites  in  the  U.S.  for  interpretation.  This  capability  will  also  be 
demonstrated  at  the  Technology,  Entertainment,  and  Design  TEDMED2  conference 
in  Charleston,  SC,  on  May  15, 1998. 

2.  Military  demonstration  and  evaluation. 

Summary  description  (as  submitted  by  Dr.  Wally  Smith  for  DARPA  internal 
"highlights"  list,  February  18, 1998): 

“Ultrasound  Telepresence  to  be  Demonstrated  At  Sea. 

The  MUSTPAC  development  team  led  by  Battelle  is  currently  working  with  the 
U.S.  Navy  to  place  a  MUSTPAC  ultrasound  telepresence  system  for  at-sea  testing 
on  the  aircraft  carrier  Enterprise  when  it  deploys  in  Oct/Nov  1998,  probably  to  the 
Mediterranean.  Prior  to  deployment  on  the  Enterprise,  the  MUSTPAC  will  reside 
at  the  National  Naval  Medical  Center,  Bethesda  MD,  for  a  period  of  3-6  months, 
for  evaluation,  development,  and  vetting  in  the  Navy's  operational  environment. 
NNMC,  an  integrated  medical  team  composed  of  radiologists,  surgeons,  nurses, 
and  other  medical  providers  will  develop  protocols,  collect  and  evaluate  clinical 
data,  and  train  the  intended  users.  During  deployment,  ultrasound  scans  will  be 
performed  on  the  Enterprise  by  non-radiologists  trained  NNMC  to  use  the 
equipment,  and  will  be  transmitted  to  NNMC  evaluation  by  radiologists  there. 
This  effort  was  initiated  and  is  being  coordinated  by  CAPT  Rick  Bakalar,  MD, 
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USN,  Head  of  Telemedicine  Department  at  NNMC,  as  requested  by  VADM 
Koenig.  Clinical  studies  are  being  coordinated  by  CAPT  Chuck  Maori,  MD, 
USN.” 

3.  Demonstration  at  IEEE  Ultrasonics  Symposium  (Sendai,  Japan). 

The  MUSTPAC,  accompanied  by  one  project  team  member,  will  be  demonstrated  as 
an  "invited  poster"  at  the  1998  international  IEEE  Ultrasonics  Symposium. 

In  FY98,  FY99,  and  FYOO,  progress  toward  the  deliverables  described  above  was  as 

follows: 

1 .  The  Everest  Extreme  Expedition  test  and  associated  TEDMED2  Conference 
presentation  were  highly  successful,  despite  a  formidable  collection  of  technical, 
administrative,  and  logistic  hurdles  to  be  overcome.  In  the  end,  a  ruggedized 
MUSTPAC™  configuration  based  on  a  Fieldworks  FW7666P  rugged  laptop  and  an 
Ausonics  Impact  VFI  ultrasound  scanner  were  taken  to  Mt.  Everest  Base  Camp  by 
Dr.  Christian  R.  Macedonia  (MAJ,  US  Army  MC),  co-inventor  of  the  MUSTPAC™ 
system.  At  Base  Camp  Dr.  Macedonia  performed  a  total  of  32  volumetric  (3-D) 
scans  and  transmitted  these  to  MUSTPAC™  diagnostic  workstations  positioned  at 
Yale  University  and  Walter  Reed  Army  Medical  Center  (WRAMC). 
Communications  were  accomplished  using  a  sat-com  based  electronic  mail  facility 
provided  by  Jim  Bruton,  the  expedition  logistics  leader.  In  addition  to  the  3-D  scans, 
Dr.  Macedonia  also  collected  a  total  of  292  still  images  (JPEG  format)  containing 
Doppler  blood  flow  information  relating  to  physiological  adaptation  to  altitude. 
These  results  were  discussed  on  May  15, 1998,  during  a  real-time  videoconference 
between  Base  Camp,  WRAMC,  and  the  TEDMED2  conference  in  Charleston,  SC. 
See  (Appendix  K)  for  pictures  of  the  Everest  configuration.  (Medical  scientific 
results  from  the  Everest  mission  will  be  published  separately  by  Dr.  Macedonia.) 

2.  The  National  Naval  Medical  Center  (NNMC)  demonstration  and  evaluation  was 
repeatedly  delayed  due  to  a  combination  of  administrative  and  staff  availability 
factors.  From  a  technical  standpoint,  progress  was  acceptable,  with  an  initial 
installation  at  NNMC  on  May  15, 1998,  followed  by  upgrades  for  increased 
functionality  on  June  5  and  July  16.  However,  NNMC’s  clinical  investigation 
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protocol  was  not  approved  by  the  Institutional  Review  Board  (IRB)  at  NNMC  until  3 
Mar  1999,  over  a  year  after  the  study  was  first  discussed  and  agreed  upon  in 
principle.  As  a  result  of  these  administrative  delays,  there  was  no  progress  on  the 
actual  clinical  investigation  during  FY98,  and  very  little  during  FY99.  In  late  FY99, 
a  further  no-cost  time  extension  through  the  end  of  FYOO  was  negotiated  to  give 
NNMC  more  opportunity  to  complete  this  study.  However,  the  study  continued  to  be 
impeded  by  staffing  changes  and  low  priority  at  NNMC.  Furthermore,  the  potential 
relevance  of  this  study  to  future  MUSTPAC™  applications  became  progressively  less 
clear,  since  the  NNMC  study  was  defined  to  use  only  the  motor-driven  linear  scanner 
instead  of  the  more  recent  freehand  capability  (see  item  #5,  below).  However,  the 
very  long  time  required  for  initial  approval  of  the  protocol  argued  against  changing  it 
at  any  point.  In  addition,  the  medical  personnel  involved  were  more  comfortable 
with  the  concept  of  parallel  scanning  and  preferred  to  have  that  constraint  enforced 
by  the  motor-driven  scanner.  By  July  2000,  it  became  clear  that  there  was  no  further 
value  in  formally  continuing  the  NNMC  investigation,  so  by  agreement  between 
NNMC,  Battelle,  and  Dr.  Wally  Smith  (DARPA  project  manager),  the  investigation 
was  terminated  with  no  further  activity. 

3.  Demonstration  at  the  1998  IEEE  International  Ultrasonics  Symposium,  and 
subsequent  publication  of  results  was  accomplished  on  schedule.  See  (Appendix  L), 
R.  J.  Littlefield,  C.R.  Macedonia,  J.R.  Coleman,  MUSTPAC™  3-D  Ultrasound 
Telemedicine  System,  1998  IEEE  International  Ultrasonics  Symposium,  Sendai, 
Japan,  Oct.5-8, 1998. 

For  FY98-00,  other  notable  activities  and  results  (some  negative)  are  as  follows: 

4.  Conversion  of  the  MUSTPAC™  system  to  Windows  NT  was  completed. 

5.  MUSTPAC™  now  includes  a  high  quality  freehand  scan  capability,  utilizing  a  high 
precision  mechanical  arm  as  a  6-degree-of-ffeedom  position  sensor.  This  was 
accomplished  by  integrating  into  MUSTPAC™  a  new  acquisition  and  display 
software  package  called  3D  FreeScan,  a  product  of  EchoTech  3D  Imaging  Systems 
GmbH  (Hallbergmoos,  Germany),  then  convincing  EchoTech  to  modify  their 
software  to  acquire  positioning  information  using  a  high  precision  mechanical  arm 
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(the  MicroScribe  arm  from  Immersion  Corp.,  San  Jose,  CA)  instead  of  a  lower 
precision  free-space  magnetic  sensor.  In  addition  to  greatly  improving  data  quality, 
use  of  the  mechanical  arm  also  obviates  earlier  concerns  about  the  impact  on 
MUSTPAC™  of  the  active  EMF  cancellation  used  by  Navy  ships.  Freehand  scan 
capability  using  the  mechanical  arm  was  delivered  to  Dr.  Chris  Macedonia  at 
Georgetown  University  in  mid-November  1998,  for  use  in  Dr.  Macedonia’s  study  on 
amniotic  fluid  and  cervical  length  measurements. 

6.  The  TelelnViVo  software  from  Fraunhofer  CRCG  has  been  replaced  as  the  software 
of  choice  by  the  3D  FreeScan  software  from  EchoTech  (see  previous  item).  At  some 
future  time,  it  is  expected  that  TelelnViVo  will  be  removed  altogether  from 
MUSTPAC™. 

7.  The  MUSTPAC™  and  Dr.  Christian  Macedonia  appeared  before  the  House 
Commerce  Committee,  Subcommittee  on  Health  and  Environment,  June  5,  1998. 

8.  A  possible  application  of  MUSTPAC™  for  prostate  cancer  detection  &  monitoring 
was  identified.  Dr.  Chris  Macedonia  presented  the  MUSTPAC™  at  the  70th 
Semiannual  Meeting  of  the  Japanese  Society  of  Ultrasonics  in  Medicine  (JSUM) 
(Sendai,  Japan,  Nov.  1997).  While  in  Sendai,  Dr.  Macedonia  collaborated  with  Dr. 
Yoshikatsu  Tanahashi  to  utilize  the  MUSTPAC™  to  reprocess  video  data  recorded 
during  an  intraurethral  prostate  examination,  producing  a  3-D  reconstruction  and 
enabling  subsequent  reslicing  at  arbitrary  angles.  This  resulted  in  a  second 
presentation,  by  Dr.  Tanahashi,  at  the  71st  Semiannual  Meeting  of  JSUM, 
highlighting  MUSTPAC™  images  generated  by  Richard  J.  (Rik)  Littlefield  of 
Battelle  /  Pacific  Northwest  National  Laboratory  (PNNL)  from  videotapes  sent  to  him 
by  Dr.  Tanahashi.  On  this  same  topic,  Rik  Littlefield  attended  the  workshop 
“Transperineal  Brachytherapy  for  Early  Stage  Prostate  Cancer”  held  by  Northwest 
Hospital  (Seattle,  WA,  Jan.12-13,  1998). 

9.  A  possible  application  for  MUSTPAC™  in  screening  for  thyroid  nodules  among 
Chernobyl  victims  was  identified  by  Dr.  Richard  Satava,  previously  with  DARPA 
and  currently  at  Yale.  This  application  was  not  pursued  due  to  lack  of  funding. 
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10.  The  MUSTPAC™  patent  application,  filed  June  25, 1997,  was  rejected  on  the 
grounds  of  obviousness  compared  against  patent  #5,609,485,  "Medical  Reproduction 
System",  by  Mark  Bergman  et.al.,  granted  Mar.l  1, 1997  and  assigned  to  MedSim, 
Ltd.  of  Israel.  This  patent  was  unknown  to  the  project  team  at  the  time  of  the 
MUSTPAC™  filing  (and,  obviously,  MUSTPAC’s  earlier  development). 

Fortunately,  the  MedSim  patent  specifically  restricts  its  claims  to  the  application 
domain  of  training,  so  the  existence  of  this  patent  apparently  does  not  create  any 
issues  for  use  of  MUSTPAC™  in  medical  practice. 

1 1.  No  commercialization  partner  has  been  identified,  although  several  letters  of  interest 
were  received  in  response  to  a  CBDNet  notice  published  in  March  1999. 

12.  Several  individual  technology  improvements  developed  or  identified  by  this  project 
have  been  transferred  into  the  commercial  domain.  These  include: 

•  The  “virtual  ultrasound  probe”  user  interface  specification,  adopted  by  EchoTech 
for  their  3D  FreeScan  product. 

•  Use  of  the  high  precision  mechanical  arm  (Immersion  MicroScribe™)  for  3D 
data  acquisition,  also  adopted  by  EchoTech  for  3D  FreeScan. 

•  Medical  quality  video  capture  by  a  PCMCIA-format  interface  card,  adopted  by 
EchoTech  for  3D  FreeScan. 

13.  At  NASA’s  invitation,  Richard  J.  (Rik)  Littlefield  presented  MUSTPAC  at  NASA’s 
Seminar  in  Ultrasound:  Applications  and  Implications  in  Human  Spaceflight, 
Houston,  TX,  July  25-26, 2000. 

14.  Under  separate  travel  funding,  but  utilizing  MUSTPAC™  equipment,  presentations 
were  given  at  the  following  conferences  /  workshops: 

•  Pacific  Medical  Technology  Symposium  (PacMedTek),  Honolulu,  HI,  August  18- 
21, 1998,  presented  by  Larry  Skelly,  PNNL  and  Dr.Chris  Macedonia,  US  Army 
Medical  Corps. 

•  U.S.  Dept  of  Energy  (DOE)  Biomedical  Technologies  Exposition,  Washington 
DC,  April  20, 1999,  presented  by  Richard  J.  (Rik)  Littlefield,  PNNL. 
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•  Advanced  Technology  Applications  to  Combat  Casualty  Care  (ATACCC-98), 
Fort  Walton  Beach,  FL,  November  17-19, 1998,  presented  by  Laura  Curtis, 
PNNL. 

15.  Project  review  meetings  were  held  or  attended  as  follows: 

•  Advanced  Medical  Technologies  Workshop,  San  Diego,  CA,  Jan  28-29, 1998. 

•  MUSTPAC  Expert  Panel  Review,  ISIS  Center,  Georgetown  University, 
Washington,  DC,  Feb  10, 1998. 

•  DARPA  40th  anniversary  celebration,  Arlington,  VA,  April  6,  1998. 

•  Image  Conference,  Scottsdale,  A Z,  Aug  6, 1998. 

16.  Discussions  regarding  potential  follow-on  projects  (separately  funded)  were  held  as 

follows: 

•  McKennan  Hospital  (Sioux  Falls,  SD):  August  25, 1998;  November  16,  1998; 
December  4;  1998;  April  14,  1999. 

•  South  Dakota  Congressional  briefing,  regarding  McKennan  Hospital  proposal, 
Georgetown  University  ISIS  Center,  Washington  DC,  December  9,  1997. 

•  Mercy  Hospital  (Darby,  PA):  August  31, 1998;  June  8, 1998;  February  17, 1999. 

17.  Two  follow-on  projects  for  MUSTPAC™  have  been  developed: 

•  For  McKennan  Hospital  in  Sioux  Falls,  South  Dakota,  using  a  mix  of  U.S. 
Department  of  Agriculture  and  non-government  matching  funds  arranged  by 
McKennan,  Battelle  will  develop  and  place  several  MUSTPAC™  systems  into 
McKennan's  existing  telemedicine  system.  Getting  this  project  in  place  has  been 
repeatedly  delayed  by  lack  of  non-government  funds  to  match  the  USD  A 
contribution.  As  of  this  writing  (August  2000),  this  project  is  expected  to  be 
finalized  in  September  2000. 

•  For  Mercy  Hospital  in  Philadelphia,  using  government  funding  coordinated 
through  TATRC,  Battelle  will  place  one  unit  for  clinical  evaluation,  with  follow- 
on  phases  to  place  and  evaluate  additional  units  for  emergency  care.  This  project 
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officially  began  on  June  14, 1999.  See  Appendix  M  for  a  complete  description  of 
this  project. 

IRB  References 

All  investigations  involving  human  subjects  in  this  project  have  been  performed  in 
adherence  to  applicable  policies  of  Federal  Law  45  CFR  46,  approved  and  monitored  by 
Institutional  Review  Boards  at  the  Pacific  Northwest  National  Laboratory  (PNNL)  and 
other  participating  organizations  as  follows: 

•  Everest  Extreme  Expedition:  Georgetown  University  Medical  Center  (GUMC)  IRB 

approval  #98-1 13;  PNNL  IRB  approval  #94-6. 

•  National  Naval  Medical  Center,  IRB  approval  #B98-070;  PNNL  IRB  approval 

#94-6-5. 

•  Amniotic  fluid  and  cervical  length  measurements:  GUMC  IRB  approval  #179-97; 

PNNL  IRB  approval  #94-6. 

3.  KEY  RESEARCH  ACCOMPLISHMENTS 

Numerous  research  accomplishments  have  been  described  in  detail  in  the  previous 
section.  Over  the  entire  duration  of  the  project,  key  accomplishments  can  be  summarized 
as  follows: 

•  Laboratory  demonstration  of  high  resolution  optimally  focused  3-D  ultrasound 
imaging  using  a  mechanically  simulated  dense  2-D  array  transducer  with 
computational  holography  image  reconstruction. 

•  Theoretical  development  of  methods  to  merge  computational  holography  image 
reconstruction  with  lens-based  imaging,  offering  the  potential  for  increased  depth 
of  focus  in  lens-based  systems  using  high  resolution  2-D  array  receivers  with  a 
separate  transmit  transducer. 

•  Development  and  demonstration  of  the  MUSTPAC-1  ultrasound  telemedicine 
system,  including  successful  field  evaluation  in  Bosnia  and  Germany,  leading  to 
the  1997  Discover  Award  in  Computer  Hardware  and  Electronics  and  other 
awards  and  publications. 
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•  Development  of  the  MUSTPAC-2  system  and  successful  field  demonstration  as 
part  of  the  Everest  Extreme  Expedition. 

•  Development  of  the  MUSTPAC-3  ultrasound  telemedicine  system  and  associated 
technology  transfer,  leading  to  off-the-shelf  availability  of  hardware  and  software 
for  the  3-D  scanning  and  “virtual  probe”  reconstruction  capabilities  of 
MUSTPAC. 

4.  REPORTABLE  OUTCOMES 

•  12  publications;  see  publications  section  below. 

•  1  patent  application,  number  08/882,178  dated  June  25,  1997;  see  item  #10  above. 

•  25  meetings  and  conference  presentations;  see  details  above  in  discussion  of 
project  chronology. 

•  Additional  funding  sought,  roughly  $800K  in  two  projects;  see  item  #17  above. 

5.  PUBLICATIONS 

•  Macedonia  CR.  December  4, 1998.  The  Philosophy  and  Practice  of  Medical 
Telepresence  in  Rural  Health,  Sioux  Falls  SD  (with  Satellite  Broadcast  to  the  quad 
state  region  in  cooperation  with  the  Indian  Health  Service). 

•  Macedonia,  CR.  1998.  “Telemedicine  and  ultrasonography:  making  waves,” 
Ultrasound  Obstet  Gynecol,  12:1-2.  (See  Appendix  G,  in  the  1999  Annual  Report.) 

•  Macedonia  CR,  J  Collea,  J  Sanders.  1998.  “Telemedicine  Comes  to  Obstetrics  and 
Gynecology”,  Contemp  Ob  43:92-1 11. 

•  Macedonia  C,  RJ  Littlefield,  J  Collea,  H  Landy,  R  McClaren,  K  Lewis,  I  Zalud,  L 
Bartholomew,  JC  Pezzullo,  G  Eglinton.  January,  1999.  A  comparison  of  standard 
endovaginal  cervical  length  sonography  to  free-hand  acquired  three-dimensional 
endovaginal  sonography.  Abstract  to  the  Society  of  Maternal  Fetal  Medicine 
(formerly  SPO),  San  Francisco  CA. 

•  Macedonia  CR.,  RJ  Littlefield,  J  Coleman,  RM  Satava,  T  Cramer,  G  Mogel,  and  G 
Eglinton.  1998.  “Three-dimensional  ultrasonographic  telepresence” ,  Journal  of 
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Telemedicine  and  Telecare,  4:4,  224-230.  (See  Appendix  F  in  the  1999  Annual 
Report.) 

•  Macedonia  CR,  SK  Mini,  RJ  Littlefield.  November  1997.  Telemedicine  and 
Ultrasound:  A  Comparison  of  Two-dimensional  and  Three-dimensional  Imaging 
Techniques.  70th  Semiannual  Meeting  of  the  Japanese  Society  of  Ultrasonics  in 
Medicine,  Sendai  Japan. 

•  Littlefield  RJ.  April  20, 1999.  MUSTP AC™  3-D  Ultrasound  Telemedicine  Imaging 
System,  DOE  Biomedical  Technologies  Exposition,  Washington  DC.  (See  Appendix 
N.) 

•  Littlefield  RJ,  CR  Macedonia,  JR  Coleman.  1998.  MUSTPAC™  3-D  Ultrasound 
Telemedicine/Telepresence  System,  1998  IEEE  International  Ultrasonics  Symposium, 
Sendai,  Japan,  October  5-8.  (See  Appendix  L.) 

•  Littlefield  RJ,  Heiland  RW,  Battelle,  PNNL.  1996.  Virtual  Reality  Volumetric 
Display  Techniques  for  3-D  Medical  Ultrasound,  prepared  for  Medicine  Meets 
Virtual  Reality:  4  (MMVR:  4),  San  Diego,  California,  January  17-20, 1996. 

Published  in  Medicine  Meets  Virtual  Reality:  4  /  Health  Care  in  the  Information  Age, 
IOS  Press,  pp.  461-470.  (See  Appendix  B.) 

•  Sheen  DM,  Collins  HD,  Gribble  RP,  Battelle  PNNL.  1996.  Wideband  Holographic 
3-D  Ultrasonic  Imaging  of  Breast  and  Liver  Phantoms,  prepared  for  Medicine  Meets 
Virtual  Reality:  4  (MMVR:  4),  San  Diego,  California,  January  17-20, 1996. 

Published  in  Medicine  Meets  Virtual  Reality:  4  /Health  Care  in  the  Information  Age, 
IOS  Press,  pp.  461-470.  (See  Appendix  A.) 

•  Tanahashi,  Yoshikatsu,  M.D.,  Ph.D.,  Christian  Macedonia,  M.D.  May, 1998. 
Formatted  and  Retrievable  Imaging  Evaluation  for  Neoplasm  and  Disease  Screening 
High-resolution  Intraurethral  Prostagrams  (FRIENDSHIP),  presented  at  the  71st 
Semiannual  Meeting  of  the  Japanese  Society  of  Ultrasonics  in  Medicine,  Yokahama 
Japan. 

•  Dr.  Christian  R.  Macedonia.  June  5, 1998.  Testimony  before  the  House  Commerce 
Committee,  Subcommittee  on  Health  and  Environment. 
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6.  SUMMARY  AND  CONCLUSIONS 


The  goal  of  this  project  was  to  develop  a  field  prototype  medical  imaging  system 
utilizing  3-D  volumetric  ultrasound.  This  goal  was  pursued  with  a  combination  of  1) 
research  on  holographic  techniques  using  2-D  sensor  technology  and  computational 
reconstruction,  and  2)  development  of  clinically  usable  and  field-testable  telemedicine 
systems  based  on  mechanical  scan  of  conventional  1-D  sensor  arrays.  The  primary 
research  product  was  the  MUSTPAC™  (Medical  UltraSound,  Three-dimensional, 
Portable,  with  Advanced  Communications).  MUSTPAC™  is  an  ultrasound  telemedicine 
system  that  allows  a  patient  to  be  scanned  at  any  convenient  location  by  an  operator  with 
limited  training  and  no  ultrasound  diagnostic  skills,  then  have  those  scans  analyzed  by  a 
diagnostic  expert  at  a  center  of  excellence  at  any  location.  This  technology  may  have 
widespread  application  in  both  military  and  civilian  rural  and  emergency  healthcare, 
resulting  in  both  improved  quality  of  care  and  reduced  costs.  Several  uncertainties 
remain,  one  of  which  is  that  the  medical  community  still  needs  to  determine  what  types 
of  medical  conditions  can  be  effectively  addressed  by  MUSTPAC™.  In  any  case, 
acceptance  will  ultimately  be  determined  by  a  complex  interplay  of  issues  involving 
economics,  infrastructure,  and  regulation,  which  are  beyond  the  scope  of  study  and  effect 
for  this  project.  This  project  was  technically  successful,  but  the  technology  is  not  yet 
widely  accepted,  and  it  is  still  an  open  question  whether  MUSTPAC™  or  any  other 
ultrasound  technology  will  be  widely  adopted  for  telemedicine. 

7.  PERSONNEL  RECEIVING  PAY 

The  following  employees  of  Battelle  /  Pacific  Northwest  National  Laboratory  charged 
against  this  project  during  its  span  of  5  calendar  years: 


Management 
John  Deffenbaugh 
Sheila  Stade 
Jerry  Posakony 
Michael  Lind 
Stanley  Neuder 
Sam  Stevens 
Larry  Skelly 
Earl  Heister 
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Program  Manager 


Technial  Resource  Manager 
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APPENDIX  A 


Appendix  A:  Wideband  Holographic  3-D  Ultrasonic  Imaging 
of  Breast  and  Liver  Phantoms,  prepared  for  Medicine  Meets  Virtual  Reality: 
4  (MMVR:  4),  San  Diego,  California,  January  17-20, 1996.  Published  in 
“Medicine  Meets  Virtual  Reality:  4  /  Health  Care  in  the  Information  Age”, 
IOS  Press,  1996,  pp.  461-470. 


Reprinted  with  permission. 


Wideband  Holographic  3-D  Ultrasonic  Imaging 
of  Breast  and  Liver  Phantoms 


David  M.  Sheen  (dm_sheen@pnl.gov) 
H.  Dale  Collins  (hd_collins@pnl.gov) 
R.  Parks  Gribble  (rp_gribble@pnl.gov) 
Battelle,  Pacific  Northwest  Laboratories 


Abstract 

Wideband  ultrasonic  3-D  holography  is  a  unique  technique  for  volumetric  imaging  with  extremely 
high  lateral  and  depth  resolution.  The  large  frequency  bandwidth,  which  is  typically  25%  to  100%, 
provides  excellent  depth  resolution.  The  synthetic  aperture  provides  optimum  lateral  resolution  of 
one-half  wavelength  at  the  pulse  center  frequency.  Wideband  impulse  holography  is  a  multi¬ 
frequency  detection  and  imaging  technique  where  the  reflected  target's  broadband  time  waveform 
signals  are  recorded  over  a  defined  aperture.  The  signals  are  then  decomposed  into  their  discrete 
frequency  components  as  single  frequency  holograms,  combined  in  the  spatial  frequency  domain,  and 
reconstructed  into  a  3-D  composite  image.  The  composite  image  may  then  be  viewed  with  stereo 
glasses  in  3-D.  Recent  3-D  holographic  images  of  human  female  breast  and  liver  phantoms  with 
internal  cysts  (5  to  10  mm)  at  5  MHz  illustrate  the  efficacy  of  this  technique  for  medical  ultrasound  3- 
D  volumetric  imaging.  A  video  of  the  breast  and  liver  phantoms  may  be  presented  at  the  meeting. 


1.  Introduction 

Ultrasonic  medical  imaging  systems  presently  in  use  form  two-dimensional  images  of  slices  (B-scans) 
through  the  human  body.  These  systems  are  very  useful,  particularly  since  they  are  real-time  and  form 
many  images  per  second.  This  allows  the  operator  to  quickly  scan  through  volumes  in  the  body 
looking  for  the  anatomical  features  of  interest. 

Three-dimensional  ultrasonic  imaging  offers  significant  benefits  over  conventional  B-scan  imaging 
systems  for  medical  systems.  One  benefit  is  that  a  complete  scan  could  be  obtained  quickly  from  the 
volume  of  interest,  with  analysis  performed  in  detail  at  a  later  time.  This  analysis  could  include  3-D 
rendering  with  rotation,  tilt,  as  well  as  the  display  of  image  slices  through  the  3-D  image.  Some  of 
these  image  slices  could  be  selected  at  angles  that  are  not  possible  with  conventional  B-scanners  due 
to  the  lack  of  a  suitable  acoustic  window  through  the  body  at  that  angle.  These  tools  could  aid  in  the 
analysis  of  the  images  and  subsequent  medical  diagnosis.  Another  benefit  is  that  true  3-D  imaging 
could  be  high-resolution  in  all  three-axes.  B-scan  systems  are  typically  high-resolution  only  in  the  two 
axes  displayed  in  the  image,  with  the  elevation  direction  lower  resolution  due  to  the  height  of  the 
transducer  array. 

Three-dimensional  image  acquisition  requires  that  ultrasonic  data  be  gathered  over  two  lateral 
dimensions  with  the  time  axis  related  to  the  third  image  axis.  This  data  can  be  acquired  using  focused 
transducers,  or  using  unfocused  transducers  combined  with  computer  processing  to  focus  the  data. 

Current  real-time  B-scan  systems  use  curved  or  linear  arrays  of  transducers  which  are  time-delayed  to 
form  a  scanning  focused  beam.  This  data  is  focused  on  transmit  and  receive  and  requires  little  or  no 
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computer  processing  to  form  the  image.  This  type  of  focusing  could  be  extended  to  three-dimensions 
by  scanning  or  rotating  the  B-scan  array.  This  technique  would  not  provide  optimal  resolution  in  the 
direction  of  the  scan  since  only  a  fixed  focus  would  be  available  along  that  axis.  A  more  sophisticated 
way  to  modify  the  time-delay  focused  systems  to  3-D  would  be  to  fabricate  a  2-D  array  of  transducers, 
with  time  delay  focusing  along  both  axes.  Such  arrays  are  currently  not  available  with  the  large 
numbers  of  elements  required  for  medical  use. 

Wideband  holography  is  another  technique  that  may  be  used  to  focus  ultrasonic  image  data.  This 
technique  gathers  the  data  by  using  an  unfocused  (spherically  diverging)  beam  with  the  image  formed 
entirely  in  post-processing  using  computer  processing  to  form  the  images.  This  technique  can  be  used 
to  form  images  of  the  highest  possible  resolution,  since  each  point  in  the  three-dimensional  image  is 
focused  optimally.  It  is  not  required  to  separate  the  image  space  into  distinct  focal  zones.  Wideband 
holography  is  similar  to  traditional  acoustic  holography  [Goodman;  Hildebrand  and  Haines; 
Hildebrand  and  Brenden],  with  the  extension  to  full  wideband  processing  for  true  depth  resolution.  A 
three-dimensional  image  will  require  that  the  diverging  beam  transducer  be  scanned  over  a  two- 
dimensional  aperture,  or  that  a  linear  array  be  scanned  over  a  linear  aperture.  A  2-D  transducer  array 
could  be  used  to  form  images  with  no  mechanical  scanner  required.  However,  suitable  arrays  are  not 
yet  available.  For  the  experimental  results  presented  in  Section  4,  a  single  transducer  was  scanned 
over  2-D  aperture  to  obtain  the  3-D  image  data. 

At  this  stage,  our  research  has  been  directed  at  obtaining  3-D  ultrasonic  images  of  the  highest 
resolution  using  medical  imaging  phantoms.  As  presently  implemented,  our  experimental  techniques 
require  more  time  than  would  be  available  for  practical  medical  imaging  systems.  Ongoing  efforts  are 
directed  at  modifying  these  techniques  to  allow  for  fast  data  acquisition  and  processing.  This  work  is 
part  of  an  ARPA-sponsored  research  project  directed  at  the  development  of  3-D  ultrasonic  imaging 
systems  useful  for  the  diagnosis  of  battlefield  injuries. 


2.  Wideband  Holographic  Imaging  Technique 

Wideband  holographic  imaging  is  a  means  of  forming  high  resolution  three-dimensional  images  of 
targets  from  wide-bandwidth  data  gathered  over  a  two-dimensional  planar  aperture.  The  target  to  be 
imaged  is  illuminated  over  a  planar  aperture  using  an  unfocused,  or  diverging,  coherent  source  such  as 
an  ultrasonic  transducer.  The  transmitted  diverging  beam  impinges  on  the  target  resulting  in  echoes 
which  are  recorded  by  the  transducer.  The  reflected  or  echoed  signal  is  then  recorded  coherently 
(digitized)  and  stored  in  a  computer  memory.  The  complete  set  of  unfocused  data  can  then  be  used  to 
mathematically  reconstruct  a  fully  focused  three  dimensional  image  of  the  target's  reflectivity 
function. 

A  highly  efficient  algorithm  has  been  developed  at  Battelle  that  may  be  used  to  perform  this  image 
reconstruction.  This  technique  relies  extensively  on  the  use  of  Fourier  Transforms  which  may  be 
computed  very  efficiently  using  the  Fast  Fourier  Transform  algorithm.  The  reconstruction  method 
allows  for  wide  relative  bandwidths  and  allows  for  the  targets  to  be  near  to  the  scanned  aperture.  No 
far-field  approximations  are  made.  Therefore,  the  only  limitation  on  the  resolution  obtained  is  the 
diffraction  limited  resolution  imposed  by  the  wavelength,  source  and  receiver  beamwidths,  size  of 
aperture,  and  distance  to  the  target.  This  method  is  similar  to  single  frequency  holography  [Goodman; 
Hildebrand  and  Haines;  Hildebrand  and  Brenden],  with  the  extension  to  wideband  illumination, 
which  allows  for  true  three-dimensional  high-resolution  imagery  from  a  two-dimensional  planar 
aperture. 


A-3 


Data  collection  is  performed  by  scanning  a  transmitting  source  and  receiver  over  a  rectilinear  planar 
aperture  which  has  one  or  more  targets  within  its  field  of  view.  The  image  reconstruction  algorithm 
described  in  this  section  is  an  extension  of  work  by  Soumekh  [1991,  1994].  Soumekh  derived  a 
wideband  synthetic  aperture  imaging  algorithm  which  reconstructs  data  from  a  linear  aperture  into  a 
two-dimensional  image.  This  work  extends  this  derivation  by  making  the  aperture  planar  instead  of 
linear  which  allows  for  a  fully  three-dimensional  image  reconstruction. 

A  full  mathematical  derivation  of  the  reconstruction  algorithm  is  beyond  the  scope  of  this  paper; 
however,  an  overview  of  the  technique  will  be  given.  The  transducer  is  assumed  to  produce  a 
diverging  beam.  At  a  modest  distance  from  the  transducer,  this  beam  is  approximately  a  spherical 
wave.  This  spherical-wave  can  be  mathematically  decomposed  into  a  superposition  of  plane-wave 
components  emanating  from  the  source  over  all  possible  angles.  The  plane-wave  decomposition  is 
performed  in  the  frequency  domain,  so  the  recorded  time-domain  data  are  Fourier  Transformed  to 
obtain  the  frequency  domain  data.  After  plane-wave  decomposition  the  spherical  wave  phase  function 
is  expressed  in  terms  of  a  superposition  of  linear  phase  functions.  These  linear  phase  functions  are 
plane  waves  and  may  be  grouped  as  Fourier  integrals  and  inverted  using  multi-dimensional  Fourier 
Transforms.  Thus,  an  efficient  image  reconstruction  is  possible  based  almost  entirely  on  the  Fourier 
Transform.  For  computer  implementation,  the  Fast  Fourier  Transform  algorithm  is  used  resulting  in  a 
very  efficient  image  reconstruction  algorithm. 

The  expected  lateral  resolution  for  the  wideband  holographic  imaging  system  is  approximately  one- 
half  wavelength  times  the  F-number.  The  F-number  is  equal  to  the  range  to  the  target  divided  by  the 
aperture  width,  or  is  determined  by  the  beamwidth  of  the  transducer.  The  longitudinal  range 
resolution  is  determined  by  the  pulse  width  or  bandwidth  of  the  system,  which  are  inter-related.  The 
range  resolution  may  be  expressed  as  v/2B,  where  v  is  the  velocity  of  sound  in  the  medium,  and  B  is 
the  bandwidth  of  the  system.  The  range  resolution  may  also  be  expressed  as  vT/2  where  T  is  the  pulse 
width  in  time.  For  a  wideband  system,  both  the  lateral  and  range  resolutions  obtained  are  on  the  order 
of  one-half  to  four  wavelengths  at  the  center  frequency. 


3.  Experimental  Imaging  System 

An  experimental  imaging  system  was  assembled  to  gather  wideband  holographic  ultrasonic  data  from 
various  medical  ultrasonic  phantoms.  This  system  consists  of  a  high-resolution  two-axis  mechanical 
scanner,  a  water  bath,  an  ultrasonic  transducer  assembly  operating  at  5  MHz  with  associated  transmit 
and  receive  circuitry,  and  a  computer  with  A/D  converter. 

The  mechanical  scanner  is  high-resolution  and  can  scan  up  to  15  cm  on  each  of  two  axes.  The  scanner 
holds  the  transducer  a  short  distance  above  the  target .  The  water  bath  allows  the  ultrasonic  waves  to 
be  coupled  to  the  target,  while  still  maintaining  a  planar  scanned  aperture.  An  attractive  feature  of  the 
water-bath  system  is  that  the  shape  of  the  target  is  undisturbed  and  can  be  imaged  along  with  the 
underlying  features  of  the  target.  This  is  not  possible  using  conventional  B-scanning  techniques  in 
which  the  scanner  is  directly  coupled  to  the  target  and  therefore  does  not  accurately  image  the  shape  of 
the  target. 

The  transducer  assembly  consists  of  two  small  (60  mil)  circular  flat  transducers  operating  at  5  MHz. 

At  this  frequency  the  transducers  are  approximately  4  wavelengths  wide,  and  produce  a  beamwidth  of 
approximately  14  degrees.  This  beamwidth  results  in  an  F-number  approximately  equal  to  4.  An  F- 
number  of  4  will  produce  lateral  resolution  of  approximately  2  wavelengths.  For  the  imaging  results 
presented  in  the  next  section,  the  transmitted  waveform  consisted  of  8  wavelengths  at  5  MHz.  This 
results  in  a  depth  resolution  of  approximately  4  wavelengths,  or  twice  the  lateral  resolution  dimension. 
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The  transmitted  waveform  (8  cycles)  has  a  bandwidth  of  approximately  25%.  At  5  MHz  the 
wavelength  of  ultrasound  in  water  is  approximately  0.3  mm.  This  configuration  therefore  has  a  lateral 
resolution  size  of  approximately  0.6  mm,  and  a  depth  resolution  size  of  approximately  1.2  mm.  The 
depth  resolution  could  easily  be  improved  by  reducing  the  number  of  cycles  transmitted.  This  was  not 
done  so  that  the  size  of  the  overall  data  set  would  be  more  manageable. 

The  receiver  circuitry  amplifies  the  target  echoes  prior  to  A/D  conversion.  If  digitized  directly,  the 
received  5  MHz  waveform  would  require  sampling  rates  in  excess  of  at  least  10  MHz  and  more 
typically  20  MHz  (allowing  for  4  samples  per  wavelength).  Since  the  bandwidth  of  our  system  is  only 
approximately  25%,  the  sampling  rate  can  be  reduced  considerably  by  coherently  demodulating  the 
received  signal.  This  is  done  by  mixing  it  in  quadrature  with  the  source  oscillator  (5  MHz)  used  to 
generate  the  8  transmitted  cycles,  to  yield  in-phase  (I)  and  quadrature  (Q)  demodulated  signals.  The 
full-bandwidth  of  the  system  is  25%  of  5  MHz  or  1 .25  MHz.  Thus,  the  maximum  frequency  in  each 
of  the  I  and  Q  signals  is  0.625  MHz.  The  demodulated  I  and  Q  signals  must  be  sampled  at  a  rate  of  at 
least  1.25  MHz  or  more  typically  2.5  MHz.  Our  system  was  configured  with  a  sample  rate  of  5  MHz 
to  allow  flexibility  to  vary  the  system  bandwidth  without  changing  the  data  sampling  rate. 

Once  a  complete  scan  of  the  target  has  been  digitized  and  stored  in  the  computer,  the  computer  image 
reconstruction  algorithm  discussed  in  the  previous  section  is  applied  to  the  data.  The  numerical 
reconstruction  is  performed  using  a  high-speed  arithmetic  co-processor.  The  resulting  3-D  image  data 
can  then  be  displayed  on  the  host  computer  as  2-D  slice  images,  or  processed  further  for  3-D  rendered 
images. 

4.  Imaging  Results 

To  evaluate  the  effectiveness  of  the  wideband  holographic  imaging  technique  on  targets  as  realistic  as 
possible,  ultrasound  phantoms  of  the  human  female  breast  and  human  liver  fabricated  by  ATS 
Laboratories  were  selected.  The  breast  phantom  is  of  a  type  used  to  train  physicians  for  ultrasound 
guided  biopsies.  It  contains  simulated  cysts  which  are  in  the  form  of  10  mm  diameter  spheres.  The 
liver  phantom  is  constructed  in  a  similar  fashion  to  the  breast  phantom.  A  simulated  gall  bladder  with 
gall  stones  is  included  in  the  phantom  as  well  as  simulated  cysts. 

Unlike  conventional  B-scanners,  the  output  of  the  wideband  holographic  imaging  system  is  a  fully- 
focused  3-D  image.  In  this  paper,  we  will,  however,  not  attempt  to  render  the  data  in  a  three- 
dimensional  fashion  but  will  simply  examine  various  slices  through  the  reconstructed  3-D  images. 
Two-different  orthogonal  image  slices  will  be  shown.  B-scan  slices  are  slices  through  the  data  set 
which  are  perpendicular  to  the  plane  containing  the  scanned  aperture.  C-scan  slices  are  slices  through 
the  data  set  which  are  parallel  to  the  scanned  aperture.  It  should  be  pointed  out  that  the  B-scan  view  is 
similar  to  that  from  conventional  B-scanners,  except  that  the  thickness  of  the  slice  is  much  thinner  in 
the  3-D  B-scan  view. 

The  data  presented  in  this  paper  has  also  been  rendered  in  a  3-D  manner  and  will  be  presented  in  a 
separate  paper  at  this  conference  by  R.  Littlefield,  R.  Heiland,  and  C.  Macedonia. 

Figures  1  and  2  show  image  slices  from  the  reconstructed  3-D  image  of  the  human  breast  phantom 
imaged  at  5  MHz.  This  3-D  image  was  obtained  from  an  80  mm  by  80  mm  by  27.5  mm  scan  of  breast 
phantom  placed  in  a  water  bath.  The  phantom  was  scanned  from  above,  i.  e.  the  time  axis  corresponds 
to  the  vertical  image  axis  in  Figure  1.  To  avoid  aliasing,  a  large  number  of  data  samples  were  taken 
with  512  sample  points  in  both  lateral  dimensions  and  256  samples  (at  5  MHz  sampling)  in  time.  The 
first  image  in  Figure  1  shows  the  a  cross-section  slice  of  the  breast  phantom  which  contains  only 
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background  scatterers.  The  second  image  clearly  shows  a  10  mm  spherical  simulated  cyst.  There  is 
also  some  shadowing  below  the  cyst.  The  third  image  shows  another  cyst  near  the  bottom  right.  The 
fourth  image  clearly  shows  the  outline  of  the  breast  along  with  two  cysts.  The  nipple  and  the  area  just 
to  the  right  of  the  nipple  are  relatively  bright  in  this  image.  This  is  due  to  the  normal  to  the  surface  of 
the  breast  being  aligned  with  the  transducer  which  results  in  a  specular  reflection  at  these  points.  This 
type  of  feature  would  not  be  present  if  the  transducer  were  directly  coupled  to  the  target,  as  is 
commonly  done  with  conventional  imaging.  The  fifth  image  shows  a  bright  cyst  (echogenic)  as  well 
as  a  darker  cyst  (non-echogenic). 

Figure  2  shows  C-scan  slices  from  the  reconstructed  3-D  image  of  the  human  breast  phantom.  These 
images  clearly  resolve  the  various  simulated  cysts.  Some  of  the  darker  circular  areas  are  shadows 
caused  by  bright  cysts  blocking  some  of  the  ultrasonic  energy,  while  others  are  due  to  non-echogenic 
cysts.  The  high  lateral  resolution  is  demonstrated  by  the  bright  point  scatterers  located  on  the  surface 
of  the  breast. 

Figure  3  shows  C-scan  image  slices  from  the  reconstructed  3-D  image  of  the  human  liver  phantom 
imaged  at  5  MHz.  This  3-D  image  was  obtained  from  a  75  mm  by  100  mm  by  60.5  mm  scan  of  liver 
phantom  placed  in  a  water  bath.  To  avoid  aliasing,  a  large  number  of  data  samples  were  taken  with 
5 12  sample  points  in  both  lateral  dimensions  and  256  samples  (at  5  MHz  sampling)  in  time.  The 
second  image  in  Figure  3  clearly  shows  a  non-echogenic  (dark)  spherical  cyst.  The  third,  fourth,  and 
fifth  images  clearly  show  the  simulated  gall  bladder,  along  with  several  simulated  gall  stones.  As  in 
Figure  2,  the  high  lateral  resolution  is  demonstrated  by  the  bright  point  scatterers  seen  in  several  of  the 
images  in  Figure  3. 


Figure  1.  3-D  ultrasonic  images  of  human  breast  phantom  at  5  MHz  (B-scan  view  shown). 
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Figure  2.  3-D  ultrasonic  images  of  human  breast  phantom  at  5  MHz 
(C-scan  view). 
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Figure  3.  3-D  ultrasonic  images  of  human  liver  phantom  at  5  MHz  (C-scan  view  shown). 


5.  Conclusions 

High-resolution  three-dimensional  ultrasonic  imagery  of  breast  and  liver  phantoms  has  been 
demonstrated.  The  wideband  holographic  imaging  technique  has  been  shown  to  provide  high 
resolution  simultaneously  in  both  lateral  dimensions  as  well  as  the  range  dimension.  This  high 
resolution  is  obtained  for  all  positions  throughout  the  image.  This  is  an  improvement  over 
conventional  systems  which  use  one  or  more  fixed  focal  zones  to  obtain  the  focused  image.  In 
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addition,  conventional  B-scanners  will  have  rather  poor  resolution  in  the  elevation  direction,  i.e.,  the 
direction  normal  to  the  image  plane.  Difficulties,  however,  currently  prevent  the  application  of  this 
type  of  technique  for  clinical  use.  At  high  frequencies  (such  as  5  MHz)  the  data  set  is  very  large  and 
cannot  be  reconstructed  quickly  with  low-cost  computer  systems.  Furthermore,  the  velocity  of  sound 
limits  the  number  of  pulses  that  may  be  transmitted  per  second  which  limits  the  frame  rate  of  a  3-D 
imaging  system.  Despite  these  limitations,  we  believe  that  synthetic  focusing  techniques  such  as  the 
wideband  holographic  technique,  or  variations,  will  play  a  role  in  the  development  of  3-D  clinical 
ultrasonic  imaging  systems. 
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Appendix  B:  Virtual  Reality  Volumetric  Display  Techniques  for  3-D  Medical 
Ultrasound,  prepared  for  Medicine  Meets  Virtual  Reality:  4  (MMVR:  4),  San 
Diego,  California,  January  17-20, 1996.  Published  in  “Medicine  Meets 
Virtual  Reality:  4  /Health  Care  in  the  Information  Age”,  IOS  Press,  1996,  pp. 
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Abstract 

Ultrasound  imaging  offers  a  safe,  inexpensive  method  for  obtaining  medical  data.  It  is  also  desirable  in 
that  data  can  be  acquired  at  real-time  rates  and  the  necessary  hardware  can  be  compact  and  portable.  The 
work  presented  here  documents  our  attempts  at  providing  interactive  3-D  visualization  of  ultrasound  data. 
We  have  found  two  volume  rendering  visualization  packages  to  be  quite  useful  and  have  extended  one  to 
perform  stereographic  volume  visualization.  Using  a  relatively  inexpensive  pair  of  commercial  stereo 
glasses,  we  believe  we  have  found  a  combination  of  tools  that  offers  a  viable  system  for  enhancing  3-D 
ultrasound  visualization. 


Introduction 

Ultrasound  is  the  imaging  modality  of  choice  in  many  medical  situations  because  it  provides  very  useful 
information  for  diagnosis  and  is  safe  and  inexpensive.  In  current  practice,  ultrasound  produces  only  two- 
dimensional  images,  representing  slices  through  the  body.  These  can  be  quite  difficult  to  interpret,  and 
require  the  ultrasonographer  to  mentally  integrate  the  information  to  form  an  image  of  the  suspect  area. 
Considerable  skill  is  often  required  even  to  position  the  sensor  so  as  to  acquire  a  potentially  useful  image 
or  combination  of  images. 

If  these  difficulties  could  be  addressed,  ultrasound  could  provide  a  diagnostically  valuable,  low-cost 
imaging  modality  suitable  for  situations  where  a  rapid  diagnosis  at  a  remote  site  is  needed.  Currently,  the 
Army  (through  the  ARPA  Advanced  Biomedical  Technologies  Initiative)  is  looking  to  develop  a  portable 
3-D  ultrasonic  imaging  system  for  use  in  diagnosing  battlefield  injuries.  Our  work  is  sponsored  by  that 
initiative. 

Three-dimensional  (3-D)  ultrasound  offers  the  potential  to  alleviate  the  problems  addressed  above. 
Because  3-D  volumes  show  more  context  than  2-D  slices,  it  becomes  easier  for  users  to  understand  spatial 
relationships  and  detect  abnormal  conditions.  Positioning  the  sensor  so  as  to  acquire  useful  images  is  also 
easier  with  3-D,  because  volumetric  data  can  readily  be  rotated  and  realigned  to  good  viewing  positions, 
largely  independent  of  the  original  sensor  position. 

In  order  to  make  3-D  ultrasound  a  practical  reality,  however,  improvements  are  required  in  two  areas. 

The  first  area  requiring  improvement  is  3-D  ultrasound  image  acquisition.  One  approach  is  to 
mechanically  scan  the  sensor  head  of  a  conventional  ultrasound  unit,  perpendicular  to  its  imaging  plane, 
so  as  to  produce  a  sequence  of  conventional  B-scan  images.  The  sequence  is  then  treated  as  a  3-D  image. 
This  approach  is  reasonably  easy  and  effective,  but  suffers  from  long  image  acquisition  time  (seconds) 


B-2 


and  from  relatively  poor  focus  along  one  axis  due  to  the  thickness  of  the  ultrasound  beam.  A  second 
approach  is  to  use  2-D  arrays  of  unfocused  transducers,  combined  with  holographic  or  synthetic  aperture 
reconstruction  techniques.  This  approach  potentially  offers  both  rapid  image  acquisition  and  excellent 
resolution  on  all  axes,  but  requires  a  high  computational  rate  in  order  to  run  in  real  time.  At  this  time, 
ultrasonic  holographic  array  technology  is  only  at  the  proof-of-principle  stage.  Both  approaches  are  being 
pursued  by  an  ARPA-fimded  research  project  currently  underway  as  a  collaboration  between  Battelle  and 
Madigan  Army  Medical  Center. 

The  second  area  requiring  improvement  is  3-D  volumetric  display.  Ultrasound  presents  some  difficulties 
that  are  not  found  in  other  3-D  imaging  modalities  such  as  CT  and  MRI.  Foremost  among  these  are  that 
1)  ultrasound  images  are  typically  quite  "cluttered",  with  significant  backscatter  and  intensity  variations 
occurring  throughout  tissue  volumes,  and  that  2)  because  of  image  aperture  limitations  and  because 
ultrasound  is  inherently  directional,  the  apparent  brightness  of  tissues  and  interfaces  depends  on  then- 
position  and  orientation.  Combined,  these  two  aspects  make  it  extremely  difficult  to  explicitly 
reconstruct  anatomically  correct  3-D  surfaces  from  ultrasound  images.  This  is  a  sharp  contrast  to  CT  and 
MRI,  where  explicit  surface  reconstruction  and  display  is  often  the  method  of  choice.  Thus,  in  dealing 
with  3-D  ultrasound,  we  are  faced  with  the  dilemma  that  humans  are  not  good  at  visualizing  3-D  "clouds" 
of  data,  but  that  is  exactly  what  the  imaging  process  provides.  An  excellent  article  that  presents  the  state 
of  the  art  circa  1992  is  found  in  [Bajura].  Our  work  aims  to  address  two  open-ended  problems  discussed 
in  that  work:  visual  cues  and  real-time  volume  visualization. 

We  have  had  good  success  in  producing  easily  interpreted  displays  of  3-D  ultrasound  data  through  a 
unique  combination  of  individually  standard  display  techniques,  including: 

•  stereo  display  (using  a  commercial  LCD  headset), 

•  real-time  rotation  and  rocking, 

•  nonlinear  intensity  and  opacity  maps, 

•  a  gradient-based  lighting  model,  and  simple  shadows, 

•  all  operating  on  the  original  3-D  volume  (no  explicit  surface  reconstruction). 

The  remainder  of  the  paper  is  outlined  as  follows:  in  the  next  section,  we  provide  a  general  introduction 
to  volume  visualization  and  discuss  existing  algorithms  for  visualizing  3-D  medical  data.  Section  3 
describes  our  experiences  using  volume  rendering  to  visualize  3-D  ultrasound  data.  The  last  section 
summarizes  and  discusses  plans  for  future  related  work. 


Volume  Visualization 

Volume  visualization  can  be  defined  as  a  process  that  uses  computer  graphics  to  display  structure 
contained  in  a  three-dimensional  dataset.  This  volumetric  dataset  can  be  thought  of  as  a  discretized 
function,  F,  over  some  bounded  domain.  The  domain's  topology  varies  from  one  application  to  another, 
but  quite  often  is  rectilinear: 

F(Xi,Yj,Zk),  i=l,Nx,  j=l,Ny,  k=l,Nz 

This  is  a  generalization  of  the  "cuberille"  model  [Chen]  where  the  spacings,  dx,dy,dz,  along  the  three 
orthogonal  axes  are  equal.  CT  and  MR  data  typically  have  equal  spacing  along  intra-slice  directions 
(dx=dy),  but  the  distance  (dz)  between  slices  is  often  larger  in  order  to  minimize  ionizing  radiation 
exposure.  Scanned  ultrasound  data  often  has  the  same  format.  The  remainder  of  this  section  will  be  in 
the  context  of  rectilinear  volumetric  scalar  data. 
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There  are  currently  two  general  techniques  for  visualizing  volumetric  data:  surface  rendering  and  volume 
or  direct  rendering.  They  are  inherently  different  in  both  the  underlying  algorithms  and  the  results 
produced.  We  provide  an  overview  of  the  more  popular  algorithms  in  the  remainder  of  this  section. 

The  distinction  in  the  results  produced  by  the  two  techniques  is  that  a  surface  Tenderer  typically  generates 
polygonal  geometry,  whereas  a  volume  Tenderer  generates  an  image.  Because  surface  rendering  preceded 
volume  rendering  in  the  genealogy  of  computer  graphics  algorithms,  graphics  workstations  have  evolved 
to  be  extremely  fast  polygon  Tenderers.  This,  at  least  in  part,  accounts  for  the  large  collection  of  papers 
and  software  aimed  at  surface  rendering.  Volume  rendering  has  become  increasingly  popular  since  the 
SIGGRAPH  '88  publication  of  three  papers  on  the  subject  [Sabella, Upson, Drebin].  There  seems  to  be  an 
ongoing  debate  as  to  whether  surface  rendering  or  volume  rendering  is  superior.  Little  information  is 
published  that  tries  to  make  a  comparison  between  the  two  [Udupa].  As  we  hinted  in  the  Introduction, 
certain  medical  imaging  modalities  may  lend  themselves  better  to  one  visualization  technique  over 
another. 

Probably  the  most  prevalent  algorithm  to  perform  surface  rendering  on  volumetric  data  is  Marching 
Cubes  [Lorensen].  This  algorithm  constructs  a  polygonal  isosurface  from  rectilinear  scalar  data  as 
follows: 

•  for  each  cube  (voxel)  consisting  of  eight  vertices,  determine  the  topology  of  the  through  that  cube; 

•  using  an  enumerated  case  table  of  topologies,  compute  the  geometry  of  the  triangles  the  isosurface  via 
some  interpolant; 

•  for  improved  shading,  compute  the  gradients  at  the  cube  vertices  and  then  interpolate  to  obtain 
triangle  vertex  normals. 

The  SIGGRAPH  '88  papers  mentioned  above  offer  a  glimpse  into  the  classification  of  volume  rendering 
algorithms.  The  two  broad  classes  are  image-order  and  object-order.  An  image-order  algorithm  casts 
rays  from  image  pixels  into  the  volumetric  data.  For  each  voxel  that  a  ray  intersects,  relevant  information 
is  accumulated  to  compose  the  final  pixel  value.  An  object-order  algorithm  operates  on  the  volumetric 
data  in  memory-order  and  essentially  projects  ("splats")  the  voxels  onto  the  image  in  either  front-to-back 
or  back-to-front  order.  The  front-to-back  method  is  essentially  a  volumetric  Z-buffer  algorithm.  What 
these  two  classes  of  algorithms  have  in  common  is  the  basic  functionality  of  mapping  the  scalar 
volumetric  data,  F,  into  both  an  intensity  (color)  function,  I(F(x,y,z)),  and  an  opacity  function, 

0(F(x,y,z)).  These  are  collectively  known  as  transfer  functions.  Allowing  a  user  to  (interactively) 
modify  these  functions  permits  improved  visualization  of  relevant  structure(s)  within  the  data. 

Drebin  et  al  presented  an  algorithm  for  transforming  the  shaded  volumetric  data  into  the  viewing 
coordinate  system  to  allow  for  a  projection,  via  compositing  [Porter],  along  an  axis  of  the  stored  volume. 
This  work  was  clarified  and  extended  in  [Hanrahan]  which  decomposed  the  viewing  transformation  into  a 
sequence  of  three  shearing  matrices.  A  further  improvement  to  this  basic  concept  of  aligning  voxels 
(object  space)  with  pixels  (image  space)  is  described  in  [Lacroute  94],  In  this  work,  object  space  is 
mapped  into  a  sheared  object  space  which  is  ideally  suited  for  projecting  (compositing)  into  image  space. 
Furthermore,  the  authors  take  advantage  of  spatial  coherence  by  implementing  scanline-based  data 
structures  in  both  object  space  and  image  space  in  order  to  speed  up  the  rendering  algorithm.  Moreover, 
they  offer  three  variants  of  the  algorithm: 

•  render  a  fixed,  preclassified  volume,  but  with  an  arbitrary  viewing  transform  and  arbitrary  shading 
parameters  (fastest)  {A  classified  volume  is  one  that  has  an  assigned  to  each  voxel} 
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•  render  a  fixed  volume  with  an  associated  min-max  octree;  arbitrary  classification,  viewing  transform, 
and  shading  (slower) 

•  render  a  fixed  volume  with  no  precomputed  data  structures  (slowest). 

An  implementation  of  this  algorithm  is  available  via  the  Internet  and  is  known  as  the  VolPack  library. 

Another  volume-rendering  technique  uses  3-D  texture  hardware  to  speed  up  the  projection  of  object  space 
to  image  space  [Cabral, Cullip].  One  such  architecture  capable  of  performing  this  is  the  Silicon  Graphics 
RealityEngine.  SGI  provides  an  implementation  of  such  an  algorithm,  known  as  Volren. 


Results 

We  began  our  investigation  with  the  visualization  of  ultrasound  data  from  a  breast  phantom  containing 
internal  cysts.  The  data  was  obtained  by  another  research  group  at  Battelle  using  a  holographic 
ultrasonic  imaging  technique  [Sheen],  We  were  initially  given  a  512x512x128  dataset  of  floating  point 
values  and  told  that  the  data  should  be  log  scaled  for  better  viewing.  To  make  the  data  volume  more 
manageable  we  first  averaged  each  of  the  128  slices  to  256x256  resolution. 

We  wanted  to  first  render  this  particular  dataset  using  a  volume  rendering  technique  and  therefore 
downloaded  the  Volren  package.  After  converting  our  data  into  TIFF  images,  one  of  the  required  formats 
for  Volren,  we  discovered  that  our  machine  had  only  enough  texture  memory  to  handle  a  128x128x64  (or 
equivalent  size)  volume.  We  therefore  averaged  our  128  slices  down  to  128x128  resolution  and  then 
averaged  pairs  of  slices  to  obtain  a  128x128x64  volume.  Figure  1  shows  the  resulting  volume  rendering 
as  well  as  the  Volren  graphical  user  interface  (GUI).  We  found  Volren  to  be  a  very  powerful  tool  for 
demonstrating  the  concept  of  volume  rendering  to  others.  The  GUI  allowed  for  interactive  control  of 
transfer  functions  that  we  did  not  expect  to  be  possible  at  the  beginning  of  our  project.  Combined  with 
dynamic  viewing  (rotation  and  scaling),  Volren  provided  us  terrific  initial  results.  However,  with  the 
demand  for  the  SGI  Onyx  RealityEngine  at  a  premium,  we  sought  other  avenues  for  visualizing  our 
volumetric  data. 


TUt  Qfrtboit  7«Mtv  _ "  .  ■  V^"'  ijcip 


Figure  1.  Breast  phantom  data  rendered  using  Volren. 
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To  satisfy  our  curiosity  at  how  well  a  surface  Tenderer  would  perform  on  this  same  data,  we  computed 
several  isosurfaces  of  differing  isovalues.  Figure  2  shows  one  such  surface  that  closely  resembles  the 
Volren  rendering.  (The  isosurfaces  were  computed  and  rendered  using  the  AVS  visualization  package.) 


Figure  2.  Isosurface  of  breast  phantom  data. 

Next,  we  downloaded  the  VolPack  (http://graphics.stanford.edu/software/volpack/)  volume  rendering 
library  and  began  learning  the  mechanics  of  it.  There  is  sufficient  documentation  in  the  form  of  a  user’s 
guide,  man  pages,  and  example  datasets  and  programs.  Though  we  were  no  longer  bound  by  the  texture 
memory  limit,  we  kept  the  breast  phantom  data  at  128x128x64  for  comparison.  Figure  3  shows  an  image 
rendered  using  VolPack  with  appropriately  chosen  transfer  functions  to  approximate  the  Volren  and 
isosurface  renderings. 


Figure  3.  Volume  rendering  using  VolPack. 
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Figure  4  (not  shown)  is  the  first  frame  in  an  MPEG  animation  showing  a  rotation  of  the  breast  data. 

These  images  were  generated  using  the  VolPack  library.  Note  that  depth-cueing  is  a  user-specified 
option.  [Note:  Figure  4  is  the  same  image  as  Figure  3.] 

In  Figure  5,  we  show  a  volume  rendered  image,  using  Volren,  of  the  face  of  a  fetus  with  a  cleft  palate. 
The  data  was  taken  with  a  conventional  B-scan  ultrasound  imaging  unit.  The  original  volume  consists  of 
50  slices  at  190x236  resolution.  Figure  6  is  the  first  image  of  an  MPEG  animation  generated  using 
VolPack.  Note  the  shadows  which,  in  addition  to  the  lighting  model,  serve  as  helpful  3-D  visual  cues. 
Here,  we  have  chosen  transfer  functions  that  allow  for  deeper  penetration  of  the  facial  skin  in  an  attempt 
to  visually  reveal  the  cleft  palate. 


Figure  5.  Volren  on  fetal  face. 
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Figure  6.  Fetal  face  (MPEG:  272K,  256x256) 


Figure  7  shows  the  image  from  a  volume  rendering  of  another  fetus  dataset.  Again  the  related  MPEG 
animation  was  generated  using  VolPack  and  demonstrates  its  scaling  and  translation  transformations. 


Figure  7.  Fetus  dataset  (MPEG:  107K,  256x256) 


One  of  our  goals  for  this  project  was  to  provide  stereoscopic  displays  of  volume  rendered  images.  Using 
the  VolPack  library,  we  generated  left  and  right  stereo  pairs  of  images.  From  these  we  could  provide 
either  cross-eye  stereo  pairs  or  interlaced  scanline  stereo  images  which  we  viewed  using  the  Virtual  i/o  i- 
glasses!  (TM)  shown  in  Figure  8. 


Figure  8.  The  Virtual  i.o  i-glasses! 


Figure  9  and  Figure  10  (not  shown)  are  MPEG  rotations  (fast  and  slow)  of  cross-eye  stereo  images. 
Figure  1 1  (not  shown)  is  an  MPEG  rotation  of  interlaced  scanline  stereo  images  which  can  be  properly 
viewed  wearing  a  pair  of  i-glasses!  (TM).  [Note:  Figure  10  is  not  shown  since  it  is  the  same  image  as 
Figure  9.  Figure  1 1,  while  possibly  useful  for  viewing  via  a  Web  browser,  is  not  shown  in  this  paper 
document  since  it  is  a  fuzzy  image  of  little  value  without  the  proper  viewing  hardware.] 


Figure  9.  Cross-eye  stereo  of  breast  phantom. 

In  addition  to  visualizing  ultrasound  data,  we  also  experimented  with  volume-rendering  CT  and  MR  data 
from  the  Visible  Human  Project  (http://www.nhn.nih.gov/ 

extramural_research.dir/visible_human.html).  Figure  12  shows  a  slice  of  CT  data  from  the  lower 
abdomen  with  clipping  lines  that  we  used  to  delimit  a  subregion  around  the  spine.  Extracting  a  volume  of 
data  in  this  fashion,  we  used  the  VolPack  library  to  generate  the  images  in  Figures  13  and  14.  Here  we 
have  chosen  appropriate  transfer  functions  to  highlight  the  spine  (bone)  from  the  rest  of  the  tissue. 


Figure  12.  CT  slice  with  clipping  lines. 


Figure  13.  Lower  spine  from  CT  data. 
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Figure  14.  Another  view  of  the  spine. 


Future  Work 

In  a  related  project  at  Battelle,  we  are  investigating  extensions  to  the  work  described  here.  We  have 
embedded  the  stereographic  volume-rendering  into  an  immersive  virtual  environment  that  allows  a  user  to 
interactively  view  and  perform  simple  editing  of  the  volumetric  data  [Heiland]. 

We  are  in  the  process  of  porting  the  VolPack  library  and  our  stereographic  image  application  to  a 
Macintosh  PowerPC  as  an  extension  to  the  NIH  Image  application. 

Finally,  we  plan  to  incorporate  and  experiment  with  a  parallel  version  of  VolPack  when  it  becomes 
available  [Lacroute  95]. 


Acknowledgments 

This  work  was  supported  by  funds  from  the  Advanced  Research  Projects  Agency  (ARP A),  contract 
number  DAMD17-94-C-4127.  Initial  research  in  volume  visualization  was  funded  by  a  Laboratory 
Directed  Research  and  Development  (LDRD)  project  through  the  Environmental  Molecular  Sciences 
Laboratory  (EMSL)  at  Pacific  Northwest  National  Laboratory.  Additional  support  was  through  a  LDRD 
project  in  the  Medical  Technologies  and  Systems  Initiative  at  Pacific  Northwest  National  Laboratory. 
Pacific  Northwest  National  Laboratory  is  a  multiprogram  national  laboratory  operated  by  Battelle  for  the 
U.S.  Department  of  Energy  under  Contract  DE-AC06-76RLO  1830.  The  National  Library  of  Medicine 
and  the  Visible  Human  Project  provided  CT  and  MR  data.  Finally,  we  would  like  to  thank  the  following 
individuals:  Dr.  Rick  Satava,  our  project  manager  at  ARPA,  for  his  consistent  encouragement;  Phil 
Lacroute  for  timely  replies  to  questions  about  VolPack;  Todd  Kulick  and  Brian  Cabral  for  answering 
questions  about  Volren;  and  our  colleagues  Dale  Collins,  Dave  Sheen,  and  Parks  Gribble  for  providing 
holographic  ultrasound  data. 


B-ll 


References 


Bajura  M,  Fuchs  H,  and  Ohbuchi  R.  1992.  “Merging  virtual  objects  with  the  real  world:  Seeing 
ultrasound  images.”  SIGGRAPH  '92  Conference  Proceedings  in  Computer  Graphics  26(2):203-210. 

Cabral  B,  Cam  N,  and  Foran  J.  1995.  “Accelerated  Volume  Rendering  and  Tomographic  Reconstruction 
Using  Texture  Mapping  Hardware.”  SIGGRAPH  '95  Conference  Proceedings  in  Computer  Graphics 
Annual  Conference  Series  1995,  pp.  91-98. 

Chen  LS,  Herman  GT,  Reynolds  RA,  and  Udupa  JK.  1985.  “Surface  Shading  in  the  Cuberille 
Environment.”  IEEE  Computer  Graphics  and  Applications,  December  1985:  33-43. 

Cullip  TJ  and  Neumann  U.  Accelerating  Volume  Reconstruction  with  3D  Texture  Hardware.  Tech 
Report  TR93-0027,  Univ  of  North  Carolina,  Chapel  Hill. 

Drebin  RA,  Carpenter  L,  and  Hanrahan  P.  1988.  “Volume  Rendering.”  SIGGRAPH  '88  Conference 
Proceedings  in  Computer  Graphics  22(4):65-74. 

Hanrahan  P.  1990.  “Three-Pass  Affine  Transforms  for  Volume  Rendering.”  Proceedings  of  Workshop 
on  Volume  Visualization  1990  in  Computer  Graphics  24(5):71-78. 

Heiland  RW,  May  RA,  Mauceri  S.  A  Low-Cost  Immersive  Virtual  Environment  for  Volume 
Visualization.  In  preparation. 

Lacroute  P  and  Levoy  M.  1994.  “Fast  Volume  Rendering  Using  a  Shear-Warp  Factorization  of  the 
Viewing  Transformation.”  In  SIGGRAPH  '94  Conference  Proceedings  in  Computer  Graphics,  Annual 
Conference  Series  1994,  pp.  451-458.  (Web  page  http://www-graphics.stanford.edu/papers/shear/) 

Lacroute  P.  1995.  “Real-Time  Volume  Rendering  on  Shared  Memory  Multiprocessors  Using  the  Shear- 
Warp  Factorization.”  To  appear  in  Proceedings  of  the  1995  Parallel  Rendering  Symposium. 

Levoy,  M.  1988.  “Display  of  Surfaces  from  Volume  Data.”  IEEE  Computer  Graphics  and  Applications 
8(3):29-37, 1988. 

Lorensen  WE  and  Cline  HE.  1987.  “Marching  cubes:  A  high  resolution  3D  surface  construction 
algorithm.”  SIGGRAPH  '87  Conference  Proceedings  in  Computer  Graphics  21(4):163-169. 

Neumann  U.  1993.  Volume  Reconstruction  and  Parallel  Rendering  Algorithms:  A  Comparative  Analysis. 
Ph.D.  Dissertation,  University  of  North  Carolina  -  Chapel  Hill. 

Porter  T  and  DufFT.  1984.  “Compositing  digital  images.”  SIGGRAPH  '84  Conference  Proceedings  in 
Computer  Graphics  1 8(3):253-259. 

Sabella  P.  1998.  “A  Rendering  Algorithm  for  Visualizing  3D  Scalar  Fields.”  SIGGRAPH  '88 
Conference  Proceedings  in  Computer  Graphics  22(4):51-58. 

Sheen  DM,  Collins  HD,  and  Gribble  RP.  1996.  “Ultra-Wideband  Holographic  3-D  Ultrasonic  Imaging  of 
Breast  and  Liver  Phantoms.”  In  Proceedings  of  Medicine  Meets  Virtual  Reality:  4  (these  proceedings). 

Udupa  JK  and  Hung  HM.  1990.  “Surface  versus  volume  rendering:  A  comparative  assessment.”  In 
Proceedings  of  the  First  Conference  on  Visualization  in  Biomedical  Computing.  Atlanta,  Georgia.  May 
22-25,  1990. 


B-12 


Upson  C  and  Keeler  M.  1988.  “VJBUFFER:  Visible  Volume  Rendering.”  SIGGRAPH  '88  Conference 
Proceedings  in  Computer  Graphics  22(4):59-64. 


APPENDIX  C 


Appendix  C:  Presentation  of  the  Prototype  Advanced  IMaging  System 
(AIMS)  atAUSA’95  (Association  of  the  U.S.  Army  1995  Annual  Meeting , 
Washington ,  DC,  October  1995). 


AIMS  Testbed  System  AIMS  Testbed  System 


AIMS  Testbed  System  AIMS  Testbed  System 

Major  Components  Major  Components 

Hitachi  EU-405Bplus  Hitachi  EU-405Bplus 
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Appendix  D;  “Evaluation  of  3-D  Ultrasound  Holographic  Imaging  at  1  MHz 
for  Detection  and  Visualization  of  Abdominal  Blood  Pooling  Using  a 
Mechanically  Scanned  Single  Channel  Transducer  and  Laboratory 
Phantoms”,  submitted  as  Appendix  2  of  the  1996  Annual  Project  Report. 


Appendix  D:  Evaluation  of  3-D  Ultrasound  Holographic  Imaging  at  1MHz 
for  Detection  and  Visualization  of  Abdominal  Blood  Pooling 
Using  a  Mechanically  Scanned  Single  Channel  Transducer 
and  Laboratory  Phantoms 

(Phase  1,  Task  2,  report  date  April  25, 1996) 

Task  Description 

Task  2  is  the  preliminary  experimental  evaluation  of  1  MHz  blood  pool  images  on 
phantoms,  etc.,  using  holographic  single  channel  and  “1-D”  array  systems  mechanically 
scanned  at  PNNL’s  Richland  facility  in  the  EDL  laboratory. 

The  single  channel  system  will  be  implemented  to  experimentally  verify  the  viability  of 
using  1  MHz  ultrasound  in  the  detection  and  imaging  of  blood  pool  phantoms  before 
building  the  first  test  sub-array  at  1  MHz. 

The  medical  trauma  community  (as  reported  by  Dr.  Steven  Doctor,  PNNL  scientist)  has 
successfully  used  higher  frequency  (3  to  5  MHz)  commercial  B-Scanners  to  detect  and 
image  blood  pooling  in  trauma  patients.  They  indicate  easy  detection  at  these 
frequencies  using  present  commercial  B-Scanners.  Typically,  only  one  day  is  spent 
training  for  blood  pooling  which  indicates  it  is  a  simple  test  with  a  commercial  B- 
Scanner. 

Our  task  is  to  evaluate  the  efficacy  of  using  1  MHz  in  the  detection  and  imaging  of  blood 
pooling  and  using  this  information  to  construct  a  “2-D”  array  capable  of  “3-D”  imaging 
for  a  portable  combat  field  system.  The  decision  to  use  the  lower  frequency  is  driven  by 
the  complexity  and  cost  of  building  a  high  frequency  “2-D”  transmit  /  receive  array  that 
can  be  used  in  the  combat  “3-D”  imaging  field  system.  A  simple  32  by  32  array  (~5cm 
by  5cm)  at  1  MHz  using  one  wave  length  element  requires  1024  transmit  /  receive 
channels  for  serial  switching  and  many  more  if  parallel  and  serial  switching  schemes  are 
used  for  imaging  in  real  time.  The  same  size  array  at  3  MHz  would  require 
approximately  100  by  100  elements  and  10,000  transmit  /  receive  serial  switching 
channels,  etc.  The  higher  frequency  system  would  also  have  greater  attenuation  and  with 
small  transmit  elements  (with  their  inability  to  transmit  high  power)  will  decrease  depth 
penetration  in  blood  pool  imaging.  There  are  many  complex  engineering  problems  to 
consider  when  building  a  high  frequency  “2-D”  transmit  /  receive  array  with  present  day 
technology.  The  design  and  fabrication  of  a  viable  “2-D”  imaging  array  has  challenged 
some  of  the  best  companies  and  universities  in  the  United  States,  Europe  and  Japan  and 
none  exist  today  in  commercial  medical  equipment. 

PNNL  will  construct  a  phantom  based  on  information  from  the  medical  community, 
image  at  1  MHz  using  our  holographic  single  channel  system,  and  compare  the  images 
with  a  commercial  B-Scanner  (3.5  MHz). 

The  phantom  will  consist  of  simple  foam  (stomach  layer)  and  duct  putty  (intestines) 
immersed  in  water  with  cavities  between  the  foam  and  duct  putty  that  simulate  blood 
pools.  This  assumes  the  combat  casualty  is  lying  on  his  or  her  back  and  the  blood  rises 
above  the  intestines  and  is  trapped  by  the  thick  muscle  layer  (stomach). 

Holographic  array  images  (1  MHz)  will  be  integrated  into  “3-D”  volumetric  blood  pool 
images  for  preliminary  evaluation  of  simulated  “2-D”  array  imaging  with  scanned  “  1- 
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D”  data.  These  data  will  simulate  the  results  expected  with  the  “2-D”  array  that  will  be 
used  in  the  battlefield  system. 

ARPA  and  expert  medical  staff  will  be  able  to  view  the  single  channel  phantom  blood 
pool  images  and  determine  the  viability  of  using  1  MHz  ultrasound  for  the  battlefield  “2- 
D”  array  blood  pool  imaging  system.  If  the  image  resolution  ( blood  pool  phantoms)  is 
found  insufficient  for  battlefield  diagnostics,  PNNL  will  (with  the  approval  of  ARPA) 
proceed  to  a  higher  frequency  (~2  MHz),  design  and  fabricate  the  system.  The  design 
will  follow  along  the  higher  frequency  route  with  the  appropriate  changes,  etc.  If  the 
resolution  is  sufficient  at  1  MHz,  we  will  proceed  with  design  and  fabrication  of  the  “  1- 
D”  array  and  the  “2-D”  array  combat  system. 

The  small  sub-array  (few  elements)  will  be  designed  and  fabricated  for  imaging  the  blood 
pool  phantom  and  comparing  it  with  the  single  channel  data.  Single  channel  scanned 
data,  using  a  one  wave  length  transducer,  is  the  optimum  configuration  for  high 
resolution  imaging  and  is  used  as  the  ’’gold  standard”  for  comparing  array  data.  The 
array,  with  its  many  elements  closely  packed  together,  has  various  cross-talk  and  acoustic 
coupling  paths  which  tend  to  degrade  the  image  quality  as  compared  with  a  single 
channel  system.  After  successful  single  channel  tests  at  1  MHz  the  “  1-D” ,  array  will  be 
designed  and  fabricated  for  testing. 

The  proposed  “1-D”,  array  will  consist  of  32  elements  spaced  one  wave  length  apart, 
approximately  5cm  in  length.  The  width  of  each  element  is  one  wave  length  resulting  in 
a  180  degrees  main  beam.  After  design  and  fabrication  of  the  “1-D”  linear  array,  tests 
will  be  conducted  using  the  medical  blood  pool  phantom  and  the  results  will  be  compared 
with  the  sub-array  data.  The  results  should  be  comparable  if  the  sub-array  data  is 
acceptable  for  blood  pool  imaging  at  1  MHz. 

The  “1-D”  array  when  scanned  in  one  direction  with  the  x-y  mechanical  scanner, 
(generating  many  lines  of  data)  will  simulate  a  “2-D”  array  in  non-real  time.  These  tests 
will  be  conducted  to  provide  simulated  “2-D”  array  blood  pool  imaging  data  for  the 
design  of  the  this  array  in  the  next  phase.  The  “2-D”  array  is  the  heart  of  the  portable 
combat  real  time  blood  pool  imaging  system  that  will  be  carried  by  the  field  medic  and 
used  in  diagnostic  battlefield  trauma. 

B-Scan  (Hitachi)  Simulated  Blood  Pool  Imaging  of  Phantom  at  3.5  MHz 

A  blood  pool  phantom  was  constructed  of  multiple  layers  of  foam  (same  type  as  used  in 
the  filter  of  a  Tektronix  scope  fan)  and  duct  putty  inserted  in  a  plastic  bag  (see  figure  1). 
The  use  of  foam  was  suggested  by  Dr.  Jonathan  Ophir,  University  of  Texas  medical 
school  who  is  engaged  in  research  dealing  with  emergency  room  trauma  using 
ultrasound  as  a  diagnostic  tool  for  blood  pooling.  His  wife,  Karen  Ophir,  is  a  registered 
diagnostic  sonographer  and  also  teaches  courses  in  emergency  room  trauma.  Typically, 
only  one  day  is  spent  training  for  detection  of  blood  pooling  in  trauma  patients.  The 
detection  is  relatively  simple  using  a  B-Scan  sector  array  operating  above  2  MHz. 

A  Hitachi  portable  B-Scan  with  a  3.5  MHz  linear  (sector)  array  was  used  to  image 
simulated  blood  pooling  using  the  phantom  shown  in  figure  1.  If  internal  injury  severs  a 
large  vein  or  artery  in  the  abdomen,  the  blood  rises  (assuming  the  patient  is  on  his  back) 
upward  toward  the  surface  of  his  stomach  and  is  trapped  between  the  intestines  and 
muscles  of  the  stomach.  The  blood  then  forces  the  upper  layer  (1 .3cm  foam)  to  slightly 
bulge  and  form  pockets  between  the  intestines  and  the  upper  layer. 
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(a)  (b) 

Figure  2.  Photographs  of  Duct  Putty  and  Foam  Used  in  the  Phantom 


Figure  2(a)  shows  the  sector  array  above  the  duct  putty  (in  plastic  container)  and  (b)  a  top 
view  of  the  black  colored  foam  (~  8  pores  per  cm)  that  was  used  in  constructing  the 
blood  pool  phantom  as  shown  in  figure  1 .  The  foam  is  very  coarse  and  the  large  pores 
can  be  seen  in  the  photograph. 
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Figure  3.  Blood  Pool  Phantom  Slice  Image  Geometry 

Figure  3  is  a  simple  graphical  illustration  of  the  image  display  used  in  constructing 
vertical  slice  images  (B-Scans)  of  blood  pool  phantoms  and  humans.  The  images  are  as 
if  one  cut  vertical  slices  through  the  phantom  (human)  and  then  viewed  them  as  a  side 
view  (YZ  or  XZ  planes). 

Figure  4  is  a  B-Scan  image  of  the  phantom  with  a  single  blood  pool  pocket  as  shown  by 
the  dark  area  between  the  intestines  and  the  upper  layer.  It  is  easily  seen  with  the  high 
frequency  sector  array  at  3.5  MHz.  The  intestines  appear  as  the  curved  bright  lines  and 
entrapped  gas  in  the  bowels  reflects  the  ultrasound  leaving  the  dark  area  below  the 
intestines  void  of  sound  waves. 


Figure  4.  B-Scan  Image  of  Blood  Pool  Phantom  (3.5  MHz) 
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Figure  5.  B-Scan  Image  of  Normal  (200  LB)  Human  (3.5  MHz) 

Figure  5  is  a  B-Scan  image  of  the  stomach  area  of  a  normal  (200  lbs)  male  for 
comparison  with  the  blood  pool  phantom  image.  The  dark  area  or  pocket  between  the 
intestines  and  upper  layer  is  absent  (no  blood  pooling).  The  phantom  image  appears  to 
be  a  relatively  good  model  (scattering,  etc.)  with  the  insitu  image  at  3.5  MHz.  The  actual 
intestines  appear  curved  in  the  human  as  simulated  with  the  duct  putty  in  a  plastic 
container  with  entrapped  air.  The  layers  of  muscle  between  the  stomach  and  the 
intestines  exhibit  bright  broken  lines  in  the  human  and  uniform  scattering  in  the  phantom. 
These  lines  can  be  simulated  in  the  phantom  by  the  insertion  of  thin  paper  towels  with 
entrapped  air.  The  phantom  was  altered  and  the  results  were  very  promising  showing 
similar  bright  broken  lines  in  the  muscle. 

There  are  a  few  dark  areas  above  the  intestines  in  the  normal  (200  LB)  human  (Figure  5), 
but  they  appear  to  have  scattering  centers  within  them  indicating  they  are  not  filled  with 
non-echo-genic  blood. 


Figure  6.  B-Scan  Image  of  Normal  (150  LB)  Human  (3.5  Mhz 
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Figure  6  is  the  B-Scan  image  of  the  stomach  area  of  a  normal  (150  LB)  human  in  good 
physical  condition  (very  low  fat  content).  The  bright  lines  below  the  surface  of  the 
stomach  are  more  concentrated  than  the  200  LB  human  and  the  area  above  the  intestines 
has  more  scattering.  The  thickness  of  the  fat  or  muscle  layer  is  thinner  on  the  150  LB 
human  indicating  less  fat  within  the  layer. 

Conclusion  of  Phantom  Blood  Pool  Imaging  (3.5  MHz)  using  a  Commercial  B-Scan 
with  a  Sector  Array 

Blood  pool  detection  and  imaging  at  3.5  MHz  is  viable  assuming  the  phantom  is 
representative  of  a  human  in  this  condition.  Information  from  the  trauma  medical 
community  supports  this  conclusion  and  indicates  that  it  is  relatively  easy  to  detect  blood 
pooling  using  ultrasound  above  2  MHz.  Information  of  blood  pooling  detection  and 
imaging  using  diagnostic  ultrasound  below  2  MHz  appears  to  be  unavailable.  We  believe 
the  lack  of  data  around  1  MHz  is  that  no  commercial  B-Scans  operate  at  frequencies 
below  2  MHz. 

If  the  Army’s  combat  portable  “3-D”  holographic  blood  pooling  system  is  to  be 
constructed  at  1  MHz,  then  experimental  imaging  data  should  support  the  decision  to 
reduce  the  “2-D”  array’s  complexity  and  cost  while  retaining  sufficient  lateral  resolution 
for  imaging.  The  1  MHz  array  frequency  percent  bandwidth  should  remain 
approximately  the  same  as  the  higher  frequency  systems  to  insure  adequate  depth 
resolution. 

Blood  pool  imaging  data  must  then  be  collected  using  a  laboratory  1  MHz  imaging 
system  to  verify  that  sufficient  scattering  from  the  stomach  layer  muscles  and  upper 
surfaces  of  the  intestines  will  provide  acceptable  resolution  for  blood  pool  imaging. 

Single  Channel  1  MHz  Holographic  “1-D”  Imaging  of  Blood  Pool  Phantoms 

A  laboratory  single  channel  rectilinear  scanned  1  MHz  holographic  imaging  system  was 
configured  to  image  blood  pool  phantoms.  The  tests  were  implemented  to  verify  or 
refute  the  viability  of  using  1  MHz  ultrasound  in  the  combat  field  system  employing  a 
small  “2-D”  array. 

The  blood  pool  phantom  used  in  the  3.5  MHz  tests  with  the  commercial  portable  B- 
scanner  was  set  up  in  the  laboratory  water  tank  for  single  channel  1  MHz  imaging  using  a 
focused  transducer  and  the  x-y  scanner.  Figure  7  is  a  simple  cross  sectional  diagram  of 
the  foam  duct  putty  blood  pool  phantom  with  a  focused  transducer  above  it  in  the  water 
tank.  The  blood  pool  is  simulated  by  water  pocket  between  the  upper  layer  foam  (muscle 
layer)  and  the  duct  putty  (intestines).  The  water  pocket  is  non-echo  genic  and  appears  in 
the  ultrasound  image  as  a  dark  area. 
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Figure  7.  Blood  Pool  Phantom  in  water  tank  for  1  MHz  Imaging  Tests 


Figure  8.  Holographic  “1-D”  Line  Image  of  Blood  Pool  Phantom  (1  MHz) 

Figure  8  is  a  holographic  (B-Scan)  image  (~  7.5  cm  in  length  and  depth)  of  the  phantom 
with  a  blood  pool  pocket  as  shown  by  the  dark  area  between  the  intestines  and  the  upper 
layer.  The  sample  density  is  0.5mm  in  the  horizontal  and  0.3mm  in  the  vertical.  It  is 
easily  seen  at  1  MHz  using  the  scanned  holographic  system  using  the  focused  transducer 
.  The  intestines  appear  as  the  bright  areas  (high  reflectivity)  as  a  result  of  entrapped  gas 
in  the  bowels.  The  low  frequency  appears  to  penetrate  around  the  air  entrapped  areas  of 
the  intestines  and  reflect  from  the  lower  foam  layers.  The  high  frequency  B-Scan  images 
show  dark  areas  below  the  intestines  indicating  loss  of  ultrasound  as  a  result  of  higher 
attenuation  at  these  frequencies. 

Single  Channel  1  MHz  Holographic  “1-D”  Imaging  of  Phantom  (Normal) 
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Figure  9.  Phantom  (No  Blood  Pool)  in  water  tank  for  1  MHz  Imaging  Tests 


Figure  10.  Holographic  “1-D”  Images  of  No  Blood  Pool  Phantom  (1  MHz) 

Figure  9  is  the  geometry  of  the  ultrasound  phantom  without  blood  pooling  used  in  1  MHz 
holographic  imaging  experiments.  The  phantom  is  essentially  identical  to  the  one  used 
that  simulated  blood  pooling,  but  without  the  water  cavity  between  the  upper  stomach 
layer  and  the  intestines. 

Figure  10  is  a  holographic  (B-Scan)  of  the  no-blood  pool  phantom  (7.5cm  length  and 
depth).  The  sample  density  is  0.5mm  in  the  horizontal  and  0.3mm  in  the  vertical. 

The  1  MHz  image  shows  the  absence  of  very  dark  areas  (compare  with  figures  7  &  8) 
between  the  stomach  muscle  layer  (upper  layer)  and  the  intestines  indicating  no  trauma 
from  internal  bleeding  in  this  area.  The  dark  areas  below  the  intestines  indicate  that  the 
ultrasonic  energy  is  highly  attenuated  as  a  result  of  entrapped  air  in  the  intestines.  Dark 
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areas  in  the  images  either  indicate  non-echo  genic  layers  or  loss  of  ultrasound  from 
highly  attenuative  layers  above  these  regions.  To  interpret  the  images  correctly  one  must 
know  where  the  source  position  of  the  ultrasound  beam  with  respect  to  the  x,y,z 
coordinates. 

Comparison  of  Image  Resolution  Between  Commercial  B-Scan  (3.5  MHz)  and 
Holographic  (1  MHz) 

Figure  1 1  is  a  composite  collection  of  commercial  B-Scan  (Hitachi  3.5  MHz)  and 
holographic  images  (1  MHz)  of  a  blood  pool  phantom  and  human  for  comparing  lateral 
and  depth  resolutions.  Figure  1 1(a)  is  a  B-Scan  image  of  the  blood  pool  phantom  with 
the  dark  area  (blood  pool)  between  the  upper  fat  or  muscle  layer  and  the  intestines 
(curved  lines).  Figures  1  l(b)is  the  1  MHz  holographic  image  of  the  blood  pool  phantom. 
The  lateral  resolution  at  1  MHz  is  obviously  less  than  the  three  focal  zone  B-Scanner  at 
3.5  MHz.  The  dark  area  (blood  pool)  in  the  holographic  image  (1  lb)  is  easily  identified 
and  imaged  at 
1  MHz. 

Figure  1 1(c)  is  the  B-Scan  image  (3.5  MHz)  of  a  200  LB  normal  human  (stomach  area) 
showing  bright  lines  interlaced  in  the  muscle  layer  and  the  characteristic  scallop  lines 
(intestines)  below  them. 

Figure  1 1(d)  is  the  1  MHz  holographic  image  of  a  phantom  without  blood  pooling.  The 
dark  area  between  the  stomach  layer  and  intestines  is  absent  indicating  no  trauma  from 
blood  pooling.  We  assume  that  a  combat  wound  (internal  bleeding)  in  this  area  would 
give  rise  to  significant  bleeding  in  this  cavity,  and  would  be  easily  identified  as  compared 
withthe  small  cavities  that  were  simulated  in  our  phantom. 


Figure  11.  Comparison  of  B-Scan  (3.5  MHz)  and  Holographic  (1  MHz)  Images  of 

Blood  Pool  Phantoms  and  Human 

3D  Volumetric  Visualization 
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Figures  12  and  13  illustrate  the  fully  three-dimensional  nature  of  the  holographic  image 
datasets.  Both  figures  show  a  64x64x256  image  (256  depth  planes),  obtained  by  a  1 
MHz  holographic  scan,  of  a  blood  pool  phantom  consisting  of  3  parallel  sections  of 
“intestine”. 


Figure  12  shows  a  full-screen  view  of  the  user  interface  provided  by  the  InViVo  software 
package  !.  A  volumetric  view  of  the  entire  data  set,  viewed  obliquely,  is  shown  in  the 
upper  left  quadrant,  with  a  wireframe  representation  of  the  viewing  angle  shown  in  the 
upper  right  inset.  The  other  three  quadrants  (outlined  in  red)  show  2-D  views  of  the 
holographic  dataset,  resliced  on  the  YZ,  XZ,  and  XY  orthogonal  axes.  (2-D  slices  at 
arbitrary  angles  can  also  be  obtained,  but  this  capability  is  not  illustrated  here.)  Figure  13 
shows  foil-size  volumetric  images  of  the  dataset  from  two  angles. 


At  present,  it  appears  that  the  major  value  of  volumetric  visualization  for  blood  pool 
detection  is  to  provide  a  3-D  context  within  which  to  evaluate  2-D  slice  images. 
However,  the  ability  to  reconstruct  (select)  arbitrary  3-D  slices  from  the  holographic 
datasets  should  provide  a  powerful  tool  for  interpretation. 


1  InViVo  is  a  product  of  the  Fraunhofer  Center  for  Research  in  Computer  Graphics,  Providence,  RI. 
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Figure  12. 3-D  and  orthogonal  slice  views  of  holographic  images,  as  rendered  by 
InViVo  software  package  (full  screen). 


Conclusion  of  Phantom  Blood  Pool  Imaging  (1  MHz)  using  PNNL’s  Holographic 
Imaging  System 

Blood  pool  detection  and  imaging  at  1  MHz  appears  to  be  viable  assuming  the  phantom 
is  representative  of  a  human  in  this  condition.  University  of  Texas  medical  staff  working 
in  the  field  of  trauma  gave  PNNL  directions  for  constructing  a  phantom  similar  to  those 
they  use  in  B-Scan  training  for  blood  pooling.  On  the  assumption  that  the  phantom  is 
representative  of  the  human  body  at  1  MHz,  blood  pooling  is  detectable  and  can  be 
imaged  at  this  frequency. 

If  the  Army’s  combat  portable  “3-D”  holographic  blood  pooling  system  is  to  be 
constructed  at  1  MHz,  sufficient  experimental  imaging  data  at  1  MHz  should  support  this 
important  decision.  The  next  phase  of  this  program  ( building  the  “2-D”  array)  depends 
on  the  successful  results  of  1  MHz  blood  pool  imaging  with  the  “  1-D”  array.  The  “  2- 
D”  array’s  complexity  and  cost  would  be  significantly  reduced  if  1  MHz  can  be  used. 


APPENDIX  E 


Appendix  E:  Coherent  Lens-Based  Image  Reconstruction ,  April  3,  1996, 
submitted  as  Appendix  3  of  the  1996  Annual  Project  Report : 
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Abstract 


This  paper  demonstrates  how  holographic,  or  Fourier  optic,  image  reconstruction 
techniques  may  be  applied  to  a  lens-based  coherent  imaging  system.  These 
techniques  overcome  the  limitations  of  lens-based  systems  such  as  limited  resolution 
and  very  limited  depth-of-field  (or  depth-of-focus).  Additionally,  these  techniques 
are  suitable  for  full  three-dimensional  high-resolution  imaging. 

1.  Introduction 

Efforts  are  presently  under  way  to  fabricate  large-scale  two-dimensional  arrays  for 
ultrasonic  imaging  applications.  For  example,  one  such  system  is  being  designed  by 
Loral  and  will  consist  of  a  2-D  array  of  128  by  128  elements.  The  element  spacing  is 
200  pm,  or  two-thirds  wavelength  at  the  center  frequency  of  5  MHz  Thus,  the  array 
will  be  approximately  25.6  mm  square.  Each  element  will  allow  quadrature  detection 
(I  and  Q)  of  the  scattered  wavefront.  The  present  design  will  use  coherent  external 
illumination  sources.  A  lens  (7  -  10  cm  diameter)  will  be  used  to  increase  the  image 
size  (demagnify)  so  that  the  visible  aperture  is  on  the  order  of  7  - 10  cm.  An 
additional  advantage  of  the  large  lens  is  that  it  will  capture  the  ultrasonic  energy  over 
a  larger  aperture  than  would  imaging  with  the  array  without  using  a  lens. 

Efficient  high-resolution  imaging  is  possible  using  a  2-D  array  without  using  a  lens  at 
all.  The  sampled  wavefront  can  be  ‘back-propagated,’  using  holographic  (Fourier 
optic  *)  image  reconstruction  techniques  to  form  the  image  that  existed  at  the  object 
plane.  This  image  reconstruction  technique  utilizes  the  2-D  Fourier  Transform  to 
decompose  the  scattered  wavefront  into  a  number  of  discrete  plane  wave  components. 
These  plane  wave  components  can  then  be  phase-shifted  (or  back-propagated)  to  the 
object  plane.  The  inverse  Fourier  Transform  then  yields  the  focused  image.  A  larger 
effective  image  size  can  be  obtained  by  artificially  increasing  the  size  of  the  data 
array  by  zero-padding.  This  type  of  image  reconstruction  has  also  been  extended  (at 
PNNL)  to  true  three-dimensional  imaging  by  utilizing  coherent  wide-bandwidth 
illumination.  The  I  and  Q  waveforms  can  be  digitized  over  a  time  interval  and  the 
time  series  can  then  be  Fourier  Transformed  to  yield  the  scattered  wavefronts 
magnitude  and  phase  over  a  wide  frequency  bandwidth.  This  data  can  then  be  back- 
propagated  resulting  in  a  fully  focused  3-D  image  of  the  object. 
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Advantages  of  this  type  of  synthetic  processing  are  that  no  paraxial,  Fresnel,  or 
Fraunhofer  approximations  are  made.  Therefore,  the  target  objects  can  be  close  to 
the  array,  and  can  fill  the  array  aperture  without  significant  degradation  of  the 
theoretical  diffraction  limited  resolution.  This  resolution  is  approximately 


^lateral 


2^l) 
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where  X  is  the  wavelength,  R  is  the  range  to  the  target,  L  is  the  aperture  size,  u  is  the 
speed  of  wave  propagation,  and  B  is  the  bandwidth.  Generally  it  is  possible  to 
achieve  F-l  or  F-2  for  resolution  of  one-half  to  one  wavelength,  where  F-n  refers  to 
the  optical  F-number  or  RJL. 


However,  there  may  be  advantages  in  using  a  lens  in  this  type  of  imaging  system. 
The  aperture  over  which  the  acoustic  energy  is  gathered  can  be  significantly  larger 
than  the  array.  This  may  lead  to  an  enhanced  signal-to-noise  ratio.  Also,  a  lens  may 
be  used  effectively  to  magnify  or  demagnify  the  image  size.  The  techniques  shown 
in  this  paper  may  be  used  to  essentially  ‘ fine-tune  ’  the  images  obtained  from  a  lens- 
based  imaging  system  to  obtain  performance  near  the  theoretical  limits. 


2.  Lens-based  Image  Reconstruction 

Data  is  gathered  from  a  coherent  lens-based  imaging  system,  as  depicted  in  Figure  1. 
Since  the  system  is  coherent,  the  magnitude  and  phase  of  the  complex  wavefront  that 
strikes  the  lens  are  measured  by  the  imaging  system.  If  the  lens  is  properly  designed, 
and  the  array  is  placed  at  the  correct  focal  zone,  then  the  image  of  the  target  object 
will  be  obtained  simply  by  talcing  the  magnitude  of  the  measured  signal, 

--^I2  +Q2  .  This  image  will  appear  inverted  due  to  the  image-reversing 

characteristic  of  the  lens.  If  the  lens  can  be  assumed  to  be  ‘thin,’  then  the  location  of 
the  array  will  obey  the  Gaussian  Lens  Formula  for  optimal  focus 

_L+J__  J_ 

si  f  (3) 

where /  is  the  focal  length  of  the  lens. 
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Figure  1  Lens-based  imaging  system. 


2.1  Thin-lens  requirement 

All  of  the  analysis  done  in  this  section  relies  on  the  thin-lens  approximation.  The 
critical  element  of  the  thin  lens  requirement  is  that  a  ray  entering  the  lens  at 
coordinates  (x,y)  can  be  approximated  as  leaving  the  lens  at  the  same  (x,y) 

coordinates.  This  allows  wave  propagation  through  the  lens  to  be  approximated  as  a 
simple  phase-shift  of  the  wave. 

2.2  Lens-based  imaging  simulation 

To  simulate  a  lens-based  imaging  system,  the  scattered  wavefront  from  a  test  target 
composed  of  12  points  arranged  to  form  the  letter  F  was  computed  by  superposing  the 
complex  phase  of  each  of  the  point  scatterers  over  an  (x,y)  aperture.  This  complex 

phase  is  simply  e  Wo  +0;  ~)  ~s  where  k  =  2n/X  ,  (x,y,0)  are  the  aperture 
coordinates,  and  (x5 ,ys,zs)  are  the  source  point  coordinates.  This  aperture  was 
chosen  to  be  just  on  the  object  side  of  the  lens  plane.  The  real-part  of  this  scattered 
wavefront  is  shown  in  Figure  2.  The  imaging  system  was  configured  for 
=  st  =  If. 

The  wavefront  (of  Figure  2)  was  passed  through  the  lens  by  multiplying  by  e~^^x,y^ 
where  <|)(x,y)  is  the  phase-shift  through  the  lens.  This  phase  function  was  chosen  so 
that  a  plane-wave  normally  incident  on  the  lens  aperture  would  focus  at  the  focal 
length,/  The  real-part  of  the  phase-front  after  passing  through  the  lens  is  shown  in 
Figure  3. 
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After  passing  through  the  lens,  the  phase-front  will  converge  as  it  propagates  toward 
the  array.  This  propagation  can  be  determined  using  Fourier  optics  techniques'.  The 
phase-front  can  be  2-D  Fourier  Transformed  which  decomposes  the  phase-front  into 
discrete  plane  wave  components.  These  plane  wave  components  can  then  simply  be 
phase-shifted  to  determine  their  amplitude  and  phase  at  any  other  depth,  z.  The 
actual  wavefront  at  this  depth  is  then  determined  by  inverse  2-D  Fourier  Transform  of 
the  plane-wave  components.  The  magnitude  of  the  wavefront  at  the  focal-plane  is 
shown  in  Figure  4.  The  image  is  inverted  (as  expected)  and  is  reasonably  well 
focused  with  each  discrete  point  in  the  F  clearly  resolved.  There  is,  however,  some 
undesirable  spreading  of  each  point’s  response.  This  is  perhaps  due  to  the  non- 
paraxial  nature  of  the  example  simulated. 


Figure  2  Object  phase-front  just  before  impinging  on  the  lens. 
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Figure  3  Object  phase-front  just  after  passing  through  the  lens. 


Figure  4  Image  magnitude  at  the  focal  plane. 

2.3  Correction  of  non-ideal  array  placement 

Fourier  optics  techniques  can  be  used  to  correct  for  placement  of  the  array  at 
positions  other  than  the  ideal  focal  plane.  Figure  5  shows  an  image  that  would  be 
obtained  if  the  array  were  placed  at  75%  of  the  correct  distance  from  the  lens.  The 
coherent  data  obtained  by  the  array,  can  however,  be  ‘forward-propagated’  using 
plane-wave  decomposition  to  the  correct  focal  plane  of  the  array.  This  results  in  the 
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image  shown  in  Figure  6,  which  is  essentially  identical  to  the  image  obtained  in 
Figure  4. 


Figure  5  Image  magnitude  at  75%  of  the  focal  distance. 


Figure  6  Image  magnitude  after  correction  to  the  correct  focal  distance  using  ‘forward-propagation.’ 

2.4  Back-propagation  to  the  object  plane 

A  superior  image  formation  method  to  that  discussed  in  Section  2.3  is  available. 
Instead  of  simply  forward  or  back  propagating  the  wavefront  to  the  focal  plane,  the 
wavefront  can  be  back-propagated  to  the  lens  plane,  phase-shifted  through  the  lens. 
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and  back-propagated  to  the  object  plane.  This  forms  a  virtual  image  of  the  object. 
Most  importantly,  this  technique  removes  the  effect  of  the  lens.  Therefore,  as  long  as 
the  shape  of  the  lens  is  known,  any  aberration  that  it  causes  in  the  image  can  be 
removed  by  this  process.  The  reconstructed  image  of  the  F  test  object  obtained  in 
this  maimer  is  shown  in  Figure  7.  This  image  is  tightly  focused  and  does  not  have  the 
sidelobe  artifacts  of  the  lens-only  reconstruction  shown  in  Figures  4  and  6.  Note  that 
in  Figure  7  the  image-reversal  is  not  present. 


Figure  7  Image  magnitude  after  ‘back-propagation’  with  phase-correction  for  the  lens. 

3.  Application  to  full  3-D  imaging 

The  example  demonstrated  in  Figures  2-7  was  simulated  using  a  single  frequency. 
These  techniques  are,  however,  fully  adaptable  to  wideband  3-D  imaging.  The  effect 
of  the  lens  on  each  frequency  component  of  a  wideband  system  can  be  eliminated 
using  the  technique  demonstrated  in  Section  2.4.  After  the  effect  of  the  lens  is 
removed,  the  efficient  3-D  holographic  reconstruction  techniques  (developed  at 
PNNL)  can  be  applied.  Three-dimensional  imaging  is  important  in  the  lens-based 
imaging  system,  since  it  will  allow  a  virtually  unlimited  depth-of-field  (or  depth-of- 
focus). 

The  following  results  demonstrate  the  effectiveness  of  these  techniques  for  wideband, 
three-dimensional  imaging.  For  ease  of  simulation,  the  wideband  3-D  reconstruction 
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algorithm  was  reduced  to  a  two-dimensional  case  with  a  single  cross-range  dimension 
and  one  time/range  dimension. 

The  goals  of  the  following  simulation  are  first  to  simulate  the  time-domain  data 
received  by  an  array  placed  near  the  focal  plane  of  a  lens,  and  second  to  demonstrate 
how  that  time  domain  data  can  be  reconstructed. 

It  is  most  convenient  to  simulate  the  data  first  in  the  frequency  domain.  The  complex 
wavefront  emanating  from  point  scatterers  is  determined  using  the  same  method  as 
Section  2.2.  An  example  phase-distribution  from  a  single  point  scatterer  is  shown  in 
Figure  8,  where  the  horizontal  axis  is  cross-range  and  the  vertical  axis  is  frequency. 
The  data  undergoes  a  frequency-dependent  phase  shift  when  passing  through  the  lens, 
as  shown  for  a  single  point  scatterer  in  Figure  9.  After  forward  propagation  to  the 
focal  plane,  each  frequency  component  is  focused  laterally  by  the  lens,  as  shown  in 
Figure  10.  This  frequency-domain  data  is  converted  back  to  the  time-domain  by 
inverse  Fourier  Transform.  The  amplitude  of  the  time-domain  waveforms  that  would 
be  received  by  the  array  are  shown  in  Figure  11.  Because  the  point  scatterer  was 
placed  at  the  focal  plane  and  on  the  axis  of  the  lens,  the  point  scatterer  is  imaged  with 
good  resolution  and  little  or  no  distortion. 

Wideband  reconstruction  of  the  data  shown  in  Figure  1 1,  is  accomplished  in  the 
following  manner.  The  time-domain  waveform  at  each  transducer  position  is  Fourier 
Transformed  to  form  frequency-domain  data.  The  complex  wavefront  of  each 
frequency  component  is  back-propagated  to  the  lens  and  phase-shifted  through  the 
lens.  At  this  point,  the  effect  of  the  lens  has  been  removed,  and  the  3-D  (reduced  to 
2-D  in  this  case)  FFT  based  holographic  reconstruction  algorithm  can  be  applied. 

The  reconstructed  image  of  the  single  point  scatterer  is  shown  in  Figure  12. 
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Figure  8  Object  wavefront  phase  distribution  from  a  single  point  scatterer  just  prior  to  impinging  on 
the  lens.  The  horizontal  axis  is  the  cross-range  dimension  and  the  vertical  axis  is  frequency. 


Figure  9  Object  wavefront  phase  distribution  from  a  single  point  scatterer  just  after  passing  through 
the  lens.  The  horizontal  axis  is  the  cross-range  dimension  and  the  vertical  axis  is  frequency. 
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Comparison  of  the  images  in  Figures  1 1  and  12  shows  minimal  difference  in  the 
resolution  of  the  image  of  the  point  scatterer.  For  point  scatterers  that  are  placed 
away  from  the  focal  plane  of  the  lens  and  away  from  the  axis  of  the  lens,  the 
wideband  image  reconstruction  algorithm  is  superior  to  simply  allowing  the  lens  to 
focus  the  data.  This  is  shown  clearly  in  Figure  13.  In  this  figure,  seven  point 
scatterers  were  placed  at  the  same  horizontal  position  and  at  seven  different  depths. 
The  depths  selected  were  1.3 , 1.4, 1.5, 1.6, 1.7, 1.8,  and  1.9  times  the  width  of  the 
aperture.  The  focal  plane  of  the  lens  is  at  a  depth  of  1 .6  times  the  width  of  the 
aperture  (F-1.6).  The  images  on  the  left  in  Figure  13  represent  the  simplest  image 
reconstruction  technique  of  allowing  the  lens  to  perform  all  spatial  focusing.  The 
images  on  the  right  in  Figure  13  are  reconstructed  images  obtained  using  the  full 
wideband  3-D  (reduced  to  2-D  in  this  case)  reconstruction  algorithm  described  above. 
The  seven  point  scatterers  were  centered  on  the  axis  of  the  lens  in  the  top  pair  of 
images.  The  remaining  pairs  of  images  were  offset  from  center  by  0.1,  0.2,  and  0.3 
times  the  aperture  width.  This  data  clearly  shows  the  limited  depth  of  field  present  in 
the  lens-only  based  images.  The  middle  depths  near  the  focal  plane  are  tightly 
focused,  and  the  points  away  from  the  focal  plane  are  significantly  out  of  focus.  The 
lateral  shifting  away  from  the  axis  of  the  lens  also  produces  a  distortion.  The  points 
no  longer  appear  to  be  at  the  correct  lateral  or  depth  positions.  The  data 
reconstructed  using  PNNL’s  wideband  image  reconstruction  techniques  shows 
neither  of  these  limitations.  There  is  no  apparent  distortion  of  the  positions  of  the 
point  scatterers,  and  each  point  is  imaged  with  the  diffraction-limited  resolution.  In 
fact,  the  point  scatterer  nearest  to  the  lens  show  the  highest  resolution  since  the  F- 
number  is  the  lowest  at  these  points.  This  is  despite  the  fact  that  they  are  not  near  the 
focal  plane  of  the  array. 


Figure  10  Object  wavefront  amplitude  distribution  from  a  single  point  scatterer  after  propagating  to 
the  focal  plane.  The  horizontal  axis  is  the  cross-range  dimension  and  the  vertical  axis  is  frequency. 


Figure  11  Simulated  time-domain  waveforms  from  a  single  point  scatterer  received  by  the  array.  The 
horizontal  axis  is  the  cross-range  dimension  and  the  vertical  axis  is  time/range. 


Figure  12  Image  reconstructed  from  a  single  point  scatterer  using  PNNL’s  wideband  algorithm.  The 
horizontal  axis  is  the  cross-range  dimension  and  the  vertical  axis  is  depth/range. 
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Figure  13  Comparison  of  time-domain  waveform  images  (left)  and  full  wideband  reconstructed 
images  (right)  for  seven  point  scatters  at  depths  ranging  from  1.3  to  1.9  times  the  aperture  width  at 
various  positions  across  the  aperture. 


4.  Conclusions 


Holographic,  or  Fourier  optic,  reconstruction  techniques  can  be  applied  to  lens-based 
imaging  systems.  The  only  significant  limitation  in  the  analysis  is  that  the  thin-lens 
approximation  must  be  valid.  These  techniques  have  been  demonstrated  on  simulated 
single-frequency  and  wideband  data.  The  extension  to  full  three-dimensional 
imaging  is  straightforward.  Three-dimensional  imaging  using  reconstruction 
techniques  developed  at  PNNL  allows  fully-focused  three-dimensional  imaging  with 
an  unlimited  depth  of  field  using  a  2-D  array,  with  or  without  a  lens.  If  a  lens  is  used, 
known  aberrations  that  it  causes  may  be  removed  from  the  images  algorithmically. 
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APPENDIX  F 


Appendix  F:  MUSTPAC-1: 3-D  Ultrasound  Telemedicine  System,  Web  page 
circa  October  1996. 
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MUSTPAC-1:  3-D  Ultrasound 


The  MUSTPAC  is  unique  in  that  it  provides  an  ultrasound  telemedicine  capability  that  is  effective,  requires  very  little  training  for 
anyone,  does  not  require  a  highly  skilled  operator  at  the  patient's  side,  and  operates  well  even  in  a  low-bandwidth  store-and-forward 
file  transfer  mode.  This  is  accomplished  by  1)  scanning  a  fairly  large  volume  of  the  patient  at  one  time  so  that  diagnosis  during  the 


scan  is  not  required,  and  2)  providing  the  remote  diagnostician  with  a  familiar  interface  —  a  "virtual  ultrasound  probe"  —that  mimics 
conventional  2-D  ultrasound,  so  that  their  years  of  experience  and  hand-eye  coordination  can  be  brought  to  bear  using  the  MUSTPAC 
after  only  a  few  minutes  practice. 
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The  MUSTPAC-1  prototype  weighs  approximately  85  pounds  in  its  backpack  configuration.  This  weight  includes  an  ultrasound 
system(Hitachi  EUB-905),  3D  paddle,  Indy  computer  with  camera  and  large  LCD  screen  —  everything  needed  to  acquire3D 
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Development  of  the  MUSTPAC-1  was  performed  using  "rapid  prototyping"  procedures  and  based  on  experience  with  the  predecessor 
system  developed  in  FY95.  Although  MUSTPAC-1  had  been  planned  for  several  months  earlier,  the  final  decision  to  allocate  funding 
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Appendix  G:  MUSTPAC-1:  3-D  Ultrasound  Telemedicine  Tool  for 
Deployment  Situations  in  Bosnia  and  the  European  Theater ;  Final  Report, 
"Bosnia  Task”,  PNL IRB  #94-6-1,  Project  #22258 
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ABSTRACT 

Ultrasound  is  a  popular  technology  for  medical 
imaging  of  soft  tissues  because  it  is  inexpensive,  safe, 
and  relatively  portable.  However,  the  use  of  ultrasound 
imaging  in  remote  areas  has  been  limited  because 
conventional  real-time  2-D  (two-dimensional)  ultrasound 
imaging  requires  that  a  highly  skilled  operator,  capable 
of  diagnostic  decisions,  be  present  at  the  patient’s 
location  to  point  the  data  acquisition  probe.  3-D 
ultrasound  data  acquisition  has  the  potential  to  remove 
this  limitation,  by  allowing  an  operator  with  no  diagnostic 
skills  to  collect  high  quality  scans  that  can  be  interpreted 
by  a  remote  expert.  This  capability  is  illustrated  by  the 
MUSTPAC-1,  a  portable  3-D  ultrasound  telemedicine 
system  recently  developed  for  the  U.S.  military.  In 
August  1996,  the  MUSTPAC-1  system  was  field-tested 
by  the  U.S.  Army  in  Germany  and  Bosnia.  This  report 
discusses  design  and  implementation  of  the 
MUSTPAC-1  system  and  summarizes  results  of  the 
Bosnia  field  test. 


INTRODUCTION 


Ultrasound  is  a  commonly  used  method  for 
medical  imaging  of  soft  tissues.  It  is  safe,  inexpensive, 
and  quick. 

However,  conventional  2-D  (two-dimensional) 
ultrasound  imaging  has  the  significant  drawback  of 
requiring  a  highly  skilled  operator  to  be  physically 
present  at  the  patient’s  location.  This  is  because 
conventional  2-D  ultrasound  imaging  uses  a  hands-on 
interactive  procedure  that  requires  the  operator  to  make 
diagnostic  decisions  simply  in  order  to  place  the  image 
acquisition  probe  at  the  correct  location  and  orientation 
to  see  what  needs  to  be  seen. 


For  example,  to  allow  a  diagnosis  of  gallstones 
using  conventional  2-D  ultrasound,  the  operator  must 
interactively  manipulate  the  image  acquisition  probe  so 
as  to  locate  the  gall  bladder,  image  the  bile  duct  at  the 


correct  angle  to  measure  its  diameter,  and  finally  locate 
the  stones  within  the  bladder.  A  positioning  error  of  only 
two  or  three  millimeters,  relative  to  the  patient’s  internal 
anatomy,  can  make  the  difference  between  diagnostic 
images  and  useless  ones.  This  need  for  precision 
pointing  makes  it  problematic  to  use  conventional  2-D 
ultrasound  in  a  telemedicine  setting  where  the  diagnostic 
expert  does  not  have  direct  control  over  the  probe 
positioning. 

In  contrast,  using  3-D  (three-dimensional) 
ultrasound  allows  high  quality  scans  to  be  taken  by  an 
operator  with  limited  training  and  no  diagnostic  skills. 
This  is  accomplished  by  having  the  system  scan  a  fairly 
large  volume  of  the  subject’s  anatomy  at  one  time, 
without  interpretation,  so  that  the  operator  can  use  a 
simple  “point-and-shoot”  strategy  for  data  acquisition. 

For  example,  to  scan  for  gallstones  using  3-D 
ultrasound,  the  operator  has  to  know  only  enough 
anatomy  to  scan  a  volume  that  surrounds  the  gall 
bladder.  Measuring  the  bile  duct  and  locating  individual 
stones  is  still  required,  but  this  analysis  and  diagnosis 
can  be  done  by  an  ultrasound  expert  located 
somewhere  else. 

In  the  summer  of  1996,  a  prototype  3-D 
ultrasound  telemedicine  system  was  developed  for  use 
by  the  U.S.  Army  under  field  conditions.  This  system, 
called  the  MUSTPAC-1 ,  was  tested  using  a  variety  of 
data  communication  links  between  Germany,  Bosnia, 
and  several  sites  in  the  U.S.  The  system  worked  well, 
experiencing  no  major  failures  and  exceeding 
expectations  in  some  areas. 

This  report  discusses  design  and 
implementation  of  the  MUSTPAC-1  system  and 
summarizes  results  of  the  Bosnia  field  test. 


G-2 


THE  MUSTPAC-1  SYSTEM 


OVERVIEW.  The  name  MUSTPAC-1  is  an 
acronym  for  Medical  UltraSound,  Three  dimensional  and 
Portable,  with  Advanced  Communications  -  1st 
generation.  MUSTPAC-1  is  an  ultrasound  medical 
imaging  system  that  can  scan  patients  to  generate  3-D 
volumetric  digital  datasets,  interactively  generate  3-D 
and  2-D  images  for  use  by  diagnosticians,  and  optionally 
transfer  datasets  over  standard  communication  links  to 
facilitate  remote  diagnosis  and  consultation. 

The  MUSTPAC-1  system  is  optimized  for  use  in 
a  telemedicine  framework.  It  provides  the  unique 
capability  that  high  quality  ultrasound  scans  can  be 
taken  by  an  operator  with  no  diagnostic  skills,  little 
training,  and  no  online  connection  to  an  expert. 

Typically  MUSTPAC-1  is  used  as  follows.  First, 
the  patient  is  scanned  by  placing  an  ultrasound  probe  on 
the  patient  and  mechanically  sweeping  it  across  their 
skin  over  the  area  of  interest  (Figure  1 ).  During  the 
scan,  the  system  records  ultrasound  data  from  a  sizable 
3D  volume  of  the  patient's  anatomy,  producing  a  3D 
volumetric  dataset  of  ultrasound  reflectivity.  The 
scanning  process  requires  no  interpretation  of  the 
ultrasound  images,  other  than  possibly  to  confirm  that 
the  intended  anatomy  is  covered. 


Figure  1.  Acquiring  a  3-D  abdominal  scan  using  the 
3-D  Paddle  and  actual  ultrasound  probe. 

Scans  in  the  form  of  3D  volumetric  datasets  are 
then  transmitted  over  any  standard  digital  network  to  a 
qualified  diagnostician. 

Finally,  a  diagnostician  interprets  each  3-D  scan 
using  a  Virtual  Ultrasound  Probe  that  simulates  a 
conventional  real-time  hands-on  examination  procedure. 
This  allows  the  diagnostician  to  display  arbitrary  2D 
slices  from  the  3D  dataset  simply  by  moving  the  probe 
as  if  they  were  interactively  examining  the  patient.  The 
Virtual  Ultrasound  Probe  and  corresponding  screen 
displays  are  very  natural  to  diagnosticians,  leading  to 
rapid  acceptance  and  productivity. 


Figure  2.  Interpreting  a  3-D  scan  using  the  Virtual 
Ultrasound  Probe  at  a  diagnostic  workstation. 


Figure  3.  Closeup  of  the  Virtual  Ultrasound  Probe 
and  diagnostic  workstation  display. 


Figure  4.  Actual  diagnostic  display  (human  liver 
with  clotting  in  major  veins). 
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Figure  5.  MUSTPAC-1  system  components  as 
packaged  for  military  field  evaluation. 


SYSTEM  COMPONENTS  AND  PACKAGING.  Major 
components  of  the  MUSTPAC-1  system  are  shown  in 
Figure  5.  These  components  include  (right-to-left): 

•  Backpackable  field  unit  containing  "3-D  Paddle" 
electromechanical  scanner  (at  extreme  right) 

•  Silicon  Graphics  Presenter™  flat  panel  display  (to 
right  of  backpack) 

•  Hitachi  EUB-905™  ultrasound  machine  (in 
backpack,  top  section,  with  cord) 

•  Silicon  Graphics  Indy™  computer  (in  backpack 
bottom  section) 

•  Teleconferencing  camera  (on  backpack,  top  left) 

•  Keyboard  with  integral  touchpad  (in  front  of 
backpack) 

•  High-resolution  color  monitor. 

•  Virtual  ultrasound  probe 

•  TelelnViVo™  visualization  software  (a  product  of 
Fraunhofer  CRCG,  Providence  Ri,  customized  for 
MUSTPAC-1). 

•  Other  custom  software  for  data  acquisition  and 
control. 


PRINCIPLES  OF  OPERATION.  Figure  6 
diagrams  the  major  components,  operations,  and  data 
flows  within  the  MUSTPAC-1  system. 

Scanning.  Volumetric  scanning  is  done  by 
mechanically  moving  the  probe  of  a  conventional  off-the- 
shelf  ultrasound  machine  (Hitachi  EUB-905).  As  the 
probe  moves,  the  ultrasound  machine  generates  a 
sequence  of  video  images  showing  a  2-D  slice  of  the 
subject  under  the  probe.  A  computer  digitizes  the 
sequence  of  2-D  images  and  assembles  them  to  form  a 
3-D  volumetric  dataset. 

Moving  the  ultrasound  probe  is  accomplished  by 
using  an  electromechanical  device  attached  to  it.  In  the 
MUSTPAC-1,  this  device  is  called  the  "3-D  Paddle".  It  is 
a  battery-powered  device  that  moves  the  probe  under 
motor  control,  in  a  straight  line,  at  a  precise  speed  of  1.0 
cm/second.  This  linear  scanning  movement  generates  a 
set  of  parallel  images  that  naturally  assemble  to  form  a 
dense  rectangular  3D  dataset. 

Scans  are  usually  taken  at  15  fps  (frames  per 
second),  using  a  3.5  MHz  or  5  MHz  convex-face  sector- 
scan  probe.  With  typical  penetration  depths,  this  results 
in  scanning  a  region  6  cm  wide  at  the  skin  surface  and 
20  cm  wide  at  15  cm  deep,  with  a  voxel  (volume 
element)  size  of  0.7  mm  on  each  axis. 

It  is  important  to  note  that  the  3-D  datasets 
captured  by  this  process  represent  a  sort  of  “snapshot” 
of  the  subject’s  anatomy.  Anatomical  movement  is  not 
captured,  except  in  the  form  of  motion  artifacts  that  the 
diagnostician  must  learn  to  ignore. 

In  addition  to  the  3-D  ultrasound  dataset 
generated  during  the  scan,  the  MUSTPAC-1  also 
captures  a  single-frame  image  from  each  of  two 
attached  video  cameras.  Typically  these  are  used  to 
record  placement  of  the  3-D  Paddle  on  the  subject's 
body. 


MUSTPAC-1  Principles  of  Operation 
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Visualization  for  Analysis  and  Diagnosis.  The 
MUSTPAC-1  can  generate  several  kinds  of  images: 

*  2-D  slices  at  arbitrary  positions  and 
orientations. 

*  3-D  volumes  using  "maximum  intensity 
projection"  (MIP)  and  similar  techniques  for 
combining  the  values  of  dataset  volume 
elements  (voxels). 

*  3-D  surfaces  defined  by  threshold  values  and 
rendered  using  one  of  several  shading 
techniques. 


Figure  6.  Major  components,  operations,  and  data 
flows  in  MUSTPAC-1. 


Several  images  can  be  shown  simultaneously. 
The  most  common  practice  is  to  combine  a  large  2-D 
slice  image  with  a  small  3-D  volumetric  image  (Figure  4), 
The  2-D  slice  shows  detail  that  is  useful  diagnostically, 
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while  the  3-D  volumetric  image  provides  context  by 
showing  location  of  the  2-D  slice  within  the  volume. 

Virtual  Ultrasound  Probe.  To  provide 
diagnosticians  with  a  familiar,  convenient  user  interface, 
the  MUSTPAC-1  is  equipped  with  a  Virtual  Ultrasound 
Probe  (Figure  3).  This  is  a  hand-held  6-D  input  device 
that  controls  2-D  slicing  of  volumetric  datasets  in  the 
same  way  that  a  real  ultrasound  probe  controls  2-D 
imaging  of  patients.  That  is,  theVirtual  Ultrasound  Probe 
looks,  feels,  and  acts  like  a  real  ultrasound  probe  in  that 
the  on-screen  image  is  constantly  updated  in  real  time 
(typically  5-10  times  per  second)  to  reflect  the  position 
and  orientation  of  the  probe.  (Again,  this  real  time 
updating  does  not  reflect  anatomical  movement,  which  is 
not  captured  in  the  datasets). 

In  the  MUSTPAC-1,  the  Virtual  Ultrasound 
Probe  is  implemented  using  an  Immersion  Probe™ 
(product  of  Immersion  Corporation)  with  a  dummy 
ultrasound  probe  added  to  it.  Other  6-D  input  devices, 
such  as  magnetic  free-space  trackers,  could  easily  be 
substituted  for  the  Immersion  Probe™. 

Data  Communication.  The  MUSTPAC-1  system 
provides  two  major  methods  of  data  communication: 
batch  mode  file  transmission  and  TelelnViVo™ 
incremental  transmission. 

In  batch  mode  file  transmission,  3-D  datasets 
and  associated  files  are  transferred  using  the  Internet 
standard  "ftp"  protocol.  These  transfers  are  for  ease  of 
use  over  relatively  high  bandwidth  communication 
channels.  They  are  initiated  by  the  user  from  a  "desktop" 
graphical  interface  using  a  drag-and-drop  procedure. 

Incremental  communication  capabilities  are  also 
built  into  the  TelelnViVo  visualization  program.  These 
capabilities  allow  all  or  part  of  the  full  dataset  to  be 
transferred  quickly  at  reduced  resolution,  while  the  user 
interactively  evaluates  the  images.  Typical  use  is 
envisioned  to  be  transfer  of  the  entire  dataset  with 
resolution  reduced  by  4X  per  axis  (64X  overall), 
followed  by  transmission  of  an  identified  region  of 
interest  at  the  full  resolution  of  the  dataset. 

Image  Quality.  The  MUSTPAC-1  produces 
diagnostic  quality  images. 

At  maximum  resolution,  the  saved  3-D  datasets 
capture  all  detail  present  in  the  video  output  signal  from 
the  Hitachi  EUB-905  (640x480  pixels  at  8-bit  gray  scale). 
However,  maximum  resolution  capture  often  is  not 
justified  due  to  noise  and  fuzziness  in  the  ultrasound 
imaging  process. 

Better  results  are  usually  obtained  by  a  simple 
process  that  filters  the  digitized  images  and  samples  the 
filter  output  to  construct  the  saved  3D  dataset.  Operator 
controls  are  provided  to  select  the  amount  of  filtering. 
Typical  settings  produce  one  dataset  voxel  as  the 


average  of  a  2x2  block  of  pixels  from  the  digitized  video 
image. 

There  is  no  image  degradation  due  to  storage 
and  transmission  of  the  3D  datasets.  All  transmission^ 
protocols  incorporate  error  detection  and  ™ 

correction/retransmission.  Only  non-lossy  (exact) 
compression  techniques  are  used  in  the  MUSTPAC-1. 

FIELD  EXPERIENCES.  In  August,  1996,  the 
MUSTPAC-1  system  was  deployed  by  the  U.S.  Army  to 
the  212th  Mobile  Army  Surgical  Hospital  in  Tuzla, 
Bosnia.  A  second  MUSTPAC-1  placed  at  the  Army’s 
Landstuhl  Regional  Medical  Center  in  Germany  served 
as  the  primary  receiving  station. 


Scanning.  During  the  deployment,  a  total  of  72 
scans  were  performed  on  42  volunteers,  as  follows: 


Scan  Type 

Number 

Right  Upper  Quadrant 

55 

Pelvis/Uterus/Posterior  Cul-de-Sac 

7 

Placenta 

5 

Renal 

3 

Extremity 

1 

Aorta 

1 

Total 

72 

Most  of  these  scans  were  taken  by  operators 
with  no  ultrasound  diagnostic  skills  and  minimal 
MUSTPAC-1  training  (typically  10  minutes).  No  formal 
evaluation  of  the  image  quality  has  yet  been  performed. 
Informal  evaluation  by  a  variety  of  experienced 
ultrasound  users  suggests  that  the  quality  of  the  scans 
was  largely  independent  of  an  operator’s  level  of  training 
and  generally  ranged  from  adequate  to  very  good. 

Communications.  Most  transfers  between 
Bosnia  and  Germany  were  performed  using  ftp  over  a 
T1/E1  (roughly  1  megabit/second)  geosynchronous 
satellite  link  leased  by  the  Army.  Despite  a  round-trip 
packet  delay  of  580-600  ms,  net  transfer  rates  of  roughly 
50  Kbytes/second  were  routinely  achieved.  Thus  typical 
3-D  datasets  of  6-12  Mbytes  required  only  a  few  minutes 
to  transfer  even  without  compression. 

As  exercises,  3-D  datasets  were  also 
transferred  over  two  slower  communication  links.  One 
of  these  links  was  between  Germany  and  Washington 
DC  (USA)  at  56  Kbits/sec  using  the  International 
Maritime  Satellite  System  (INMARSAT)  and  its 
associated  telephone  system  ISDN  link.  The  other  was 
between  Bosnia  and  Germany  using  the  Army’s  Tactic^^ 
Packet  Network  (TACNET)  at  9.6  Kbits/sec.  These 
transfers  were  also  successful,  but  would  be  too  slow  i^r 
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most  routine  applications  without  the  use  of  more 
aggressive  data  compression  techniques. 

Diagnostic  Usability.  During  the  Bosnia 
deployment  and  in  the  6  months  since  its  completion, 
the  MUSTPAC-1  system  has  been  operated  by 
approximately  20  experienced  ultrasound  users. 

Informal  evaluation  suggests  that  the  system  is  very 
easily  learned.  One  striking  observation  is  that  no 
experienced  ultrasound  user  to  date  has  required  more 
than  5  minutes  practice  with  the  Virtual  Ultrasound 
Probe  to  begin  making  medical  interpretations  of  3-D 
scans  that  they  had  not  previously  seen.  However,  no 
formal  studies  of  usability  or  effectiveness  have  been 
performed,  and  such  studies  will  be  needed  to  determine 
the  degree  of  diagnostic  accuracy  provided  by  the 
MUSTPAC’s  techniques. 

Adverse  Events.  During  the  field  test  in  Bosnia, 
one  subject  was  physically  pinched  by  the  3-D  Paddle, 
resulting  in  a  small  superficial  bruise  requiring  no 
medical  treatment.  The  operational  procedure  for  the  3- 
D  Paddle  was  immediately  revised  to  avoid  a  repetition, 
and  the  3-D  Paddle  has  been  redesigned  to  make 
pinching  less  likely  in  future  designs. 

LIMITATIONS.  While  MUSTPAC-1  provides  a 
unique  capability  for  ultrasound  telemedicine,  it  is  a  first- 
generation  system  with  significant  limitations.  These 
limitations  include: 

•  No  Doppler.  Neither  the  Hitachi  EUB-905 
ultrasound  scanner  nor  any  of  the  MUSTPAC-1 
software  supports  any  Doppler  capability  at  this 
time.  This  means  that  MUSTPAC-1  provides  no 
direct  visualization  of  blood  flow. 

•  Inflexible  scanning.  The  motor-driven  3-D  Paddle  is 
simple  to  use  and  robust,  but  provides  only  linear 
translation  parallel  scanning.  Freehand  scanning 
would  be  more  flexible  and  simpler  to  use  in  many 
parts  of  the  body. 

•  No  anatomic  motion.  Conventional  real-time  2-D 
ultrasound  shows  anatomic  motion,  such  as  pulsing 
arteries,  as  a  moving  image  on  screen.  This  is 
diagnostically  useful.  However,  the  3-D  scans 
captured  by  MUSTPAC-1  are  static  snapshots. 
Generally  speaking,  with  MUSTPAC-1  anatomic 
motion  produces  image  artifacts  that  the 
diagnostician  must  ignore. 

Ongoing  development  of  the  MUSTPAC  system 
is  planned  to  relieve  these  limitations. 

SUMMARY 

3-D  ultrasound  data  acquisition  can  potentially 
enable  effective  use  of  ultrasound  imaging  in  a 
telemedicine  setting,  by  allowing  an  operator  with  no 


diagnostic  skills  to  collect  high  quality  scans  that  can  be 
interpreted  by  a  remote  expert.  This  capability  is 
illustrated  by  the  MUSTPAC-1,  a  portable  3-D 
ultrasound  telemedicine  system  recently  developed  for 
the  U.S.  military  and  successfully  field-tested  by  the 
U.S.Army  in  Germany  and  Bosnia  in  August  1996. 
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APPENDIX  H 


Appendix  H:  “Ultrasound  marches  to  the  front”,  Portable  Design  Magazine 
cover  article,  June  1997. 


Reprinted  with  the  permission  of  Portable  Design  magazine,  a  PennWell 
publication. 
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Ultrasound  marches 
to  the  front 


Juki  H.  Ofirrifoikfj  i 


n  miiiuirv  medical  care,  it's  axiomatic  that 
\mi:  happens  during  ihc  golden  hour — 
rhe  60  minutes  immcdiaidv  fbljnwinix  a 


seri<  >uv  wound  or  in jury — is  key  to  pnrie.n:  t 


T 

H  survival  Unnirtiinareiy.  sopnisricarcii  di 
H  agnostic  equipment,  an  dal  to  treat  life 
threatening  wounds  swiftly,  has  been  too 
«^Bb^cumbersonic  ro  rake  into  rhe  Itcld,  and 
the  gokiai  hour  often  passes — along  with  the  case 
am:  What's  needed  is  better  portable  medical  gear 
Now,  a  joint  e  fiber  bv  researcher's  at  Pacific 
Nordve.es:  National  laboratory  (Richland,  Wnsh.i 
changes  rhe  portable  picture. 
Working  v.  H 1 1  medical  icch- 
nologv  experts  at  the  De- 
tense  Advanced 
Research  ! Vo¬ 
ice  ts  Age uc v 
(DARPAJ  and 
designers  at  the 
LhS.  Army 
Medica:  Re¬ 
search  and  Me. tend  Command,  a  de¬ 
sign  team  has  dcs'clopecl  iust  sud j  a  sys¬ 
tem  a  portable  ultrasound  imager.  It's 
purpose  is  to  hi  mg  the  bendks  of  so-. 

o  h i sheared  i il :  ra r i id  i  ma g mg ~c o- 1  - 

vcr.tior.rJiy  used  in  hospitals  10  the 
ffonr  lines  in  combat  situations. 

Commercial  potential 

Promising  to  someday  reduce  the  num¬ 
ber  of  battlefield  dearhs,  while  ottering 
tremendous  commercial  potential,  the 
joint  design  effort  resulted  in  the  design 
of  what's  called  a  Medical  Ultrasound, 

Three -di:: laisio: :a!  and  Portable  with 


"The  direction  medical  diagnostics  has  ro  go  is 
toward  making  systems  that  are  lightweight  and 
portable  in  design  so  you  can  take  them  to  the  pa¬ 
tient  A  Macedonia  h  also  a  major  with  the  l\S. 
Army  Medical-  Corps  ai  Georgetown  University 
Medical  Center,  so  he  knov\$  of  what  he  speaks. 


A  multidisciplinary 
design  team  uses  off-the- 
shelf  subsystems  to 
rapidly  develop  a  portable 
telemedicine  box. 


Ease  of  use  is  key 

The  MUSTPAC  I  permits  a  semiskilled  operator  10 
perform  A-D  scans  of  an  injured  soldier  and  send 
those  in  tapes  ibr  mlcrprctation  to  experts  anywhere 
in  rhe  world.  Macedonia  is  credited  wah  mud:  of 
The  initial  vision,  for  MUSTPAC.  In  Ian.  he  began 
exploring  the  use  of  3-D  ultrasound  while  he  was  a 
medical  student  in  die  early  1990s. 


Advanced 


.1  Comnranicarions  box 


anv 


CJ1  will  tell  you  it's  easier  to  cab  it  by  its 
uoximm:  S\ USTPAC-l). 

Mia  patient's  bleeding  in  the  lie  Id 
or,  for  that  runner,  bi  some  remote 
county  in  Wes:  Virgin  A,  I  don't  have 
rime  ro  rake  him  ro  an  MR  IT  says 
Dr.  Christian  Macedonia,  principal 
medical  investigator  on  the  projeer. 


H-2 


DEVELOPM  ENT 


STRATEGIES 


At  that  time,  Macedonia  was  interested  in  ex* 
plonng  how  different  chest  ventilation  methods  af¬ 
fected  blood  flow  throughout  die  body.  While  he 
uncovered  significant  research  on  Doppler  ultra 
, sound,  one  of  his  criticisms  was  that  ii  couldn’t  be 
used  to  track  moving  objects.  "‘It  occurred  to  me 
that  we  have  radar  systems  that  can  track  missiles  in 
flight,  so  why  can't  we  apply  some  defense  techno! 
ogy  toward  this  problem  r  ” 

It  took  a  few  years,  but  Macedonia  eventually 
hacked  together  a  prororype  that  embodied  bis 
ideas..  Using  off  the  shelf  components,  he  pieced  to¬ 
gether  a  Macintosh  7100  computer,  a  trackball  key¬ 
board,  and  an  ultrasound  machine.  lie  used  a  scan 
converter  to  transform  die  Macintosh  video  output 
to  NTSC  video  and  then  ran  it  back  out  the  moni¬ 
tor  of  the  ultrasound  machine.  Did  it  work!  ''Yes." 
tells  Macedonia,  “but  it  generated  images  that  were 
nice  to  look  au  but  weren’t  particularly  useful." 

Knowing  he'd  reached  his  technical  limits,  Mace¬ 
donia  took  his  idea  to  developers  at  Pacific  North¬ 
west  National  l  aboratory,  operated  by  Battcllc  for 
the  l US-  Department  of  Energy  (DOE).  Shortly 
thereafter,  the  DOE  and  the  Department  of  De¬ 
fense  (through  DARPA)  agreed  to  put  up  S3  million 
to  develop  a  3-D  ultrasound  system  for  diagnosing 
battle  field  injuries. 

A  MUSIPAC  must 

While  conventional  ultrasound  is  a  2-D  imaging  pro¬ 
cedure,  3-D  ultrasound  was  critical  to  the  premise 
behind  MUSTPAC — scanning  a  fairly  large  volume 
of  the  patient  at  a  remote  site  by  <ui  unskilled  oper¬ 
ator  and  then  mmsierting  the  data  in  a  srore-and-fbr- 
ward  file  transfer  mode  to  a  skilled  diagnostician- 
‘‘The  point  of  the  3-D  is  that  it  permits  data  10  be 
collected  by  someone  who  knows  a  little  anatomy, 
but  doesn’t  know  anything  about  how  to  interpret 


ultrasound  images."  explains  Rik  Littlefield,  project 
manager  at  Pacific  Northwest  National  1  aboratory. 

Littlefield  notes  that  a  classic  ultrasound  exam  is 
performed  by  a  technician  viewing  a  real  time.  13- 
framc/scc  image  as  he  moves  a  probe  around  the 
body.  Without  that  ability  and  without  the  knowl¬ 
edge  of  how  to  interpret  those  images  and  where  to 
re- point  his  hand  to  sec  what  he  really  needs  ro  see, 
the  operator  can't  perform  rhe  exam. 

“What  3-D  dara  collection  permits  is  to  collect 
perfectly  useful  data  in  a  sort  of  blind  snapshot 
mode."  says  Litricficld.  “If  you  think  someone  might 
have  a  gallstone,  obviously  the  person  who  collects 
rhe  data  has  to  know  where  the  gall  bladder  Is.  Bur 
he  doesn’t  have  to  know  what  angle  to  point  the 
probe  in  order  to  image  the  duct  nr  the  right  angle 
to  take  die  measurements." 

In  the  MUSTPAC,  a  specially-designed  simu¬ 
lated  probe  looks,  feels,  and  acts  like  a  real  ultra¬ 
sound  probe.  1  irrlcficld  adds  that  once  an  image  1$ 
stored  in  a  3-D  dataset,  it  can  later  be  reviewed  either 
locally  or  remotely. 

Fourth  of  July  prototyping 

Once  funding  was  approved  for  die  project  (thar 
happened  in  April  of  1996),  development  of  the 
MUSTPAC  proceeded  at  what  had  to  be  a  rapid 
pace.  Driving  the  ream  was  a  schedule  dial  called  for 
delivery  of  a  prototype  by  early  July 

Much  like  Macedonia’s  original  design,  MUST- 
PACM  designers  built  Lite  unit  Around  off-the-shelf 
components.  The  ream  also  imposed  a  requirement 
that  the  system  be  transportable  by  a  single  soldier. 
“As  an  officer  in  the  First  Infantry  Division  in 
Germany,  I  clearly  remember  that  the  most  useful 
equipment  was  that  which  one  person  could  carry," 
notes  Macedonia.  “If  it's  small  enough,  it  can  go  prac¬ 
tically  anywhere." 


Anatomy  of  a  telemedicine  box 
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scanner  ' 

i  A  A 


■  In  its  85-pound  backpack  configuration,  the-  y 
first-gaierahon  MUSTPAC-1  supples  what^s  f 
needed  to  acquire  3-d  ultrasound  data,  visualize 
it  locally,  and  transmit  the  data  to  a  remote  site 
'  tbr  consultahon  in  a  teleconfer^ice.  The  MUST- 
PAG-1  is  powered  from  a  commercial  uninter  ¬ 
ruptible  power  supply,  using  a  battery,  for  about 
45  minutes.  It  can  also  be  powered  from  an  ac 
line.  The  cone  of  the  system  is  a  Hitachi  Medical 
Systems  Model  £UEfSQ5  2-DultJ^sodnd  sub-'';;;;; 
system.  -  f  U' 

. ..  The  EUB-905  links  to  a  SiliaDn  Graphics  Indy 
workstation  by  means  of  a  75-ohm  video  cable. 

Mdeo-caprture  software  feeds  a  3-D  dataset  into 
.  a  volume  datafile  residing  on  the  ind/s  bard : : T 
disk.  Rendering  so^amthehp  loaj^krarilyf  j 

re-slice  the  3-D  dataset  a?  any  position  or  orientation  using  a 
starKhafone,  joystiddike  virtual  ultrasound  probei-Datafe  traris1 


f  Cflnvsn&nal  !. 

.vri-:  • 

?-D  ultrasound " 

thyi: 

..-.ftactiiris  j: 

"Video  capers 


Volume  datallle  pG 


Rendering  software: 


•1  Display  3-D  volumetric  and  2-D  slices 

-  (arbitrary  reslicing 

P'  : 


m 

Ik 

% 
m 


.  ..  ^  A  Telecommunication: 

I  lIS  f«lc  transfer  interactive  i 

collahorafico  ; 


jimmefsior*  probe) 


famed  to  remote  sites  via  a  standard  Ethernet  connection. 
■  A  stand-alone  camera  supports  vidcsoconferencinc. 
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It  was  decided  to  make  the  system  fir  into  a  hack 
pack.  “Thar  meant  it  clearly  had  ro  be  under  !  00 
pounds,"  recalls  I  irrlcfidd.  Whar  the  team  ultimately 
designed  weighed  in  at  85  pounds. 

Close  to  25  pounds  of  the  total  weight  was  attrib¬ 
utable  to  the  centerpiece  of  the  system,  an  off  the 
shelf  EUB-905  ultrasound  subsystem  tfom  Hitachi 
Medical  Systems  fTarrymwn,  NLY.).  Measuring  13.6  x 
16  x  6.4  indies  in  size  (34  X  40  x  1 6  cm)  rhe  compact 
2  D  unit  offered  a  variety  of  display  modes,  as  well  as 
1 92  channel  probes,  256  gray¬ 
scale  steps,  and  a  high-ire-  n .  . 

qucncy  probe  tltal  o per-ues  at  ; 

up  to  10  MHz.  Ir  can  also  he  |  ■»§  if 

powered  off  cither  a  120-Y  ac  |  I E  Ip  II 


oral  pounds  of  longitudinal  ti trust  and  withstand 
pushing  against  a  patient's  body. 

Weight  versus  Mips 

The  second  heaviest  component  in  the  MUSTPAC- 
1  is  its  compute  platform.  For  that  element,  the  de¬ 
sign  team  chose  an  Indy  workstation  from  Silicon 
GrupSiks  Inc.  (SGI — Mountain  View;  Calif).  Built 
around  a  64-bit  MIPS  RSO0O  micropixx:cssor,  the 
SGI  workstation  is  oprinhz.cd  for  comnutc-mtensive 


tsic  or  oattenes. 


“We  picked  the  EL5  B -905 
because  iL  offered  ;in  excellent 
combination  of  image  quality 


“A  Pentium 
portable  with 
integrated 
keyboard  will 
likely  eliminate 
the  bulk  of  the 
stand-alone 
keyboard” 


and  size  and  weight,"  explains  : 
Littlefield.  Also  important  to  ; 
die  selection  of  the  KU  B-905  . 
was  that  it  .had  won  Food  and 
Drug  Administration  FDA  : 
5IQ:ki  approval. 

With  fmurc  commerced-  \ 
izarion  on  their  minds,  j 
MUSTPAC  designers  went 
our  of  their  way  ro  avoid  cus 
romszing  the  subsystem.  “We 
avoided  like  die  plague  break¬ 
ing  inro  tkn  system,"  explains 
Littlefield,  “Doing  auyiliiitg 
at  all  to  ir  that  could  sheet  its 
performance  would  have 
voided  rhe  51G(k)  approval." 

Complementing.  the 
EUB-905  is  a  3-D  paddle 
elec tromcchani cal  scann  er. 
Compact  and  liglitweight,  the 
paddle  was  designed  to  be 
strong  enough  to  support  sev- 


IF  YOU'RE  EMBEDDING  A 
BUNCHA  BOARDS, 
TAKE  A  CLOSE  100K 


,3m 


AT  THIS. 


No.  if  s  not  a  very  large  chip,  rf s  a  very 
small  computer:  TheZF  OEModule.  It’s 
a  complete  computer  that  mounts  on 
your  board  like  a  chip. 

The*  OEMmfalr  Is  a  complete  386SX  comparer 
with  a  form  factor  just  12(1  x  :\M  x  .4*  inches 
(55.S  x  76.2  x  1 1.3  mm). 

Forget  unreliable  cables  and  vibration 
problems. 

Everytliiuffyea  need  is  built  into  the 
OEModule.  it  boots  form  it's  embedded 
BIOS  and  comes  with  DOS..  Two  COM 
pom.  a  parallel  port,  keyboard  and  speaker 
ports,  IDK  and  floppy  disk  support,  DRAM 
controller  supports  up  to  16  Megabytes,  full 
cure  logic,  even  am  internal  300K  Resident 
Flash  Disk.  It's  all  there.  Mounted  directly 
on  your  proprietary  board,  connections  are 
rock  solid  even  in  harsh  environments. 


A  familiar  development  environment. 

The  OKMndste  is  fully  PC/AT  oomparibK 
Ymr  development  tools  will  work  perfectly. 
A  complete  development  kit 
The  ZF  MicroSystfijns  SBX/386  develop¬ 
ment  board  includes  every  thing  you  need: 
hardware,  software  and  manuals.  The 
S&V886  is  compatible  with  both  AT  and 
PC/  H)4  expansion  busss.  That  means  you 
go  from  concept  to  market  in  record  time. 

Questions?  Call  us,  or  visit  our  site: 
i . n  www.zfrntcro.com 


2F  Microsystems,  Inc. 
1052  Elwell  Court 
Paic  Mg.  CA  34303 
let:  415-965-3800 
Fax:  4 15-985-4050 
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“3-jD  collected  by  someone  v?bo  knows  a 

little  ana  tom  \\  but  doesn't  know  anythin#  about  him*  to 
interpret  ultrasound  images/*  explains  Rile  Littlefields 
project  manager  at  Pacific  Northwest  National 
Laboratory. 

graphics  and  multimedia  applications.  The  MUST- 
PAG  design  team  picked  a  platform  with  a  2  Gbyte 
hard  drive  and  support  for  seven  SCSI-2  devices. 
Titc  team  then  loaded  it  with  12«X  Mbytes  of  RAM 
to  minimize  disk  accesses. 

While  bulkier  than  currently  available  laptops,  the 
Indy  v,us  deemed  a  dear  choice  lor  MUSTPAC,  as  ir 
offered  video- capture  capabilities  in  a  Unix  environ¬ 
ment*  “We  were  considering  some  Pentium- based 
notebooks,  bur  there  was  no  way  we  were  going  so 
get  the  quality  video  capture  we  needed  wirhin  the 
time  frame  we  needed  it.”  notes  I  jtriciicid.  “What's 
more,  since  our  visualization  software  already  ran  in 
Unix,  we  didn’t  have  to  do  a  port  to  Windows  NTT 

There  were  tradeoffs.  One  of  the  drawbacks  to  die 
Indy  phrrorm  was  that  it  required  a  stand  alone  key 
board.  “In  theory;  we  could've  done  die  whole  tiling 
with  a  touch  panelf1  says  LittletieLl  -Bat  we  knew  the 
users  we  were  targeting  wanted  not.  just  a  3  D  ultra¬ 
sound  system,  but  a  more  genera!  purpose  plariormT 
For  future- generation  MUSTPAC.X  T  irrldidd  indi¬ 
cates  a  Pentium  portable  with  integrated  keyboard  will 
iixClv  cumma  re  the  bulk  of  rhe  stand-alone  keyboard. 

Since  die  Indy  doesn't  come  with  an  integrated 
display,  ihe  MUSTPAC  designers  opted  to  add  an 
SGI  presenter  panel  to  ilie  backpack  configuration. 
~Wt  needed  1,024  x  1,280  pixel  resolution  with 
CRT- quality  gray  scaled'  explains  Lirtlcricld.  “Physi¬ 
cians  performing  diagnosis  at  remote  locations  use  a 
standard  analog  CRTT 

Image  objections 

“A  common  gripe  among  doctors  about  ultrasound 
captured  static  images  is  you  can't  i  ell  whai  you  Vo 
looking  at  unless  somelxxly  tells  vou  where  rite 
probe  was  pointed T  explains  lirrieficlri.  "A  standard 
joke  is  you  can't  tell  the  difference  between  an  ovar¬ 
ian  cyst  and  a  gall  bladder.” 


A  problem  for  die  MUSTPAC.’  designers  was 
the  realization  that  trained  ultrasound  diagnosti¬ 
cians  were  used  to  working  in  a  2  D  environment, 
not  with  a  3  D  dataset.  through  years  of  training, 
doctors  have  developed  great  amounts  of  skill  and 
hand  eye  ax>rdirtadon  sn  terms  of  being  able  ro  in¬ 
ter  3-D  anarorny — based  not  only  on  where  they 
think  their  hand  is  pointing  right  now,  bin  also 
how  an  image  changes  as  they  move  their  handy’ 
says  Litdendd. 

“Since  die  person  doing  rhe  diagnosis  wasn't  go¬ 
ing  to  be  physically  located  wirh  rhe  patient,  and 
wouldn't  have  direcr  control  over  die  imaging  T 
continues  Lirrletidd,  “we  frit  it  was  very  important 
to  give  rhe  operators  a  familiar  interface.  that  would 
permit  them  to  exploit  ail  their  standard  skills.” 

“Ir’s  kind  of  counter-mtuitiveT  adds  Macedonia. 
“People  who  aren’t  in  medicine  typically  think. 
'Gosh,  it  seems  like  a  3  I)  image  would  be  a  ior  more 
useful  than  a  2  D  image.'  Bur  rhe  fact  of  die  mallei 
is  that  -die  experienced  radiologist  uses  a  2-D  image 
and  does  3  I)  reconstruction  in  his  or  her  head.” 

One  answer  to  the  visualization  problem  was  sitting 
just  down  ihe  hail  in  I  Jrtlclidd's  office.  “One  day  I  was 
mulling  over  howto  solve  this  problem  and  I  happened 
ro  walk  through  our  multimedia  lab  ”  be  remembers. 
“Sitting  on  the  table  was  a  lancv  joystick,  with  XYZ 
pitch,  roly  and  yaw  Whar  more  natural  thing  to  do  dian 
to  simply  lei  the  soltwnn:  listen  to  that  device  and  put  tl  le 
corresponding  images  upon  the  screen?  YVeT  lake  a  rcc,S- 
time  examination,  1  thought  “ 

Out  of  that  idea  came  a  virtual  ultrasound,  or  im¬ 
mersion.  probe.  Offering  equivalent  6-D  sensing  ca 
pabiuty  ;  3-D  position  plus  2-D  till  and  rotation),  rhe 
probe — witli  die  help  of  special  voln  m  ^-visualization 
software  can  be  used  jo  re  slice  rhe  ultrasound  in¬ 
formation  along  arbitrary  so-called  cutting  planes. 

Software  ain't  easy 

While  rhe  MUSTPAC  >1  development  project  relied 
largely  on  off-the-shelf  components  to  minimize  do 
sign  rime,  soifwarc  was  another  story.  Much  of  the 
innovative  integration  work  for  the  MUSTPAC!- 
came  about  in  software  development. 

The  Fraunhofer  Center  I  or  Research  in  Com¬ 
puter  Graphics  (CRCG . Providence.  R.IA  played 

a  kev  role  It's  a  non  profit  computer  graphics  re¬ 
search  group  specializing  in  rhe  study  of  volume  vi¬ 
sualizations,  virtual  environments,  collaborative 
work  tools,  and  user  inxriaces 

Researcher?  at  the  center  modified  Tele) nViVo. 
3-D  ultrasound  visualization  software  based  on  In- 
Vi  Vo  3-D  volume  visualization  code  developed  h\ 
Dr.  Georgias  Sakas  at  the  insrimre’s  Darmstadt,  Ger¬ 
many,  facility.  Featuring  network  collaboration  tcx>is 
lor  remote  consultation  and  system  control,  f del; ’ 
YTVb  permits  ph\xcians  to  view  a  range  of  darn  sets, 
including  CAT  (  computer- assisted  tomography), 
MRI  i  magnetic  resonance  imaging).  MR  A  (magnetic 
resonance  angiography),  and  PET  i  positron  emission 
tomography)  scans.  It  also  allows  users  ro  exchange 
and  manipulate  the  data  sets  via  ISDN  nr  ATM  net- 
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works.  CRCG  made  modifications  of  die  software  to 
enable  die  use  of  the  \irmd  ultrasound  probe. 

CRCGT  software  used  position  and  oriemaiion 
information  derived  by  the  probe  to  re  slice  the  3-D 
ultrasound  data  along  arbitrary  earring  planes.  The 
screen  image  is  updated  in  real  time  so  that  diag¬ 
nosticians  see  the  3-D  MUSTPAC!  scans  as  comer's 
tional  2  D  images. 

GUI  anxiety 

Concurrently  with  hardware  and  TckTnViVo  inte¬ 
gration,  programmers  at  Pacific  Northwest  National 
i-aborarorv  also  wrote  code  to  bring  the  system's 
components  together.  One  of  the  more  challenging 
aspects  of  lifts  was  writing  video-capture  software 
and  a  graphical  user  interlace.  “Making  it  run  ac¬ 
ceptably  fast  was  an  ongoing  exercise  in  frustration,” 
admits  Littlefield. 

For  die  video  input  side,  MUST? AC  designers 
built  their  applications  on  top  of VI.,  a  library  from 
SGI.  But  since  VL  didn’t  handle  any  functions  on 
the  output  side,  die  team’s  programmers  had  ro 
write  custom  code  10  glue  die  VL- based  applies  •• 
dons  wirh  the  user  interface.  The  latter  was  written 
in  TCL,  a  language  and  support  library  that  conics 
out  of  the  X  Windows  world. 

While  the  actual  amount  of  code  proved  relatively 
small,  die  task  of: managing  the  video  input,  as  well  as 
handing  off  die  imagery  ro  TCL  tor  display  on  die 
screen  and  as  a  backdrop  tor  the  user  interface, 
proved  to  be  a  very  difficult  task.  “TCL  is  designed  to 
use  an  event  loop  philosophy;  k  provides  an  event 
hiuidlcrofits  own,”  Littlefield  explains.  “3GF$  VL  is 
also  designed  to  use  the  event-loop  philosophy,,  but  it 
uses  an  event  loop  of  its  own.  Getting  those  two 
glued  together  .so  tint  all  the  events  on  both  video  in¬ 
put  and  X  Windows  sides  were  properly  interleaved 
mined  into  a  matter  of  several  weeks  of  hair  tearing.” 

No  batteries  included 

Power-supply  issues  were  also  critical.  At  first,  oper¬ 
ating  power  was  not  a  concern  for  rhe  design  team. 
Intended  for  use  in  armed  forces  Mobile  Army  Sur¬ 


gical  Hospital  (MASH;  units,  the 
MUSTPAG-I  was  initially  designed  to 
operate  fn  >m  a  standard  !  20- V  no  power 
line.  However,  in  rhe  interest  of  making 
die  unit  truly  useful  in  combat,  the  sys 
tan  also  needed  a  battery-powered  op 
lion.  It  was  supplied  with  an  off- the  - 
sh elf,  30  lb  uninterruptible  power 
supply.  “The  MUST  PAT!  is  sufficiently 
low  power  that  we  could  gcr  45  minutes 
of  continuous  operation  off  a  standard 
UPS  sysrenv  says  Littlefield 

Similarly,  the  system's  eoimnumcations  options 
were  significantly  narrowed  when  initial  require¬ 
ments  called  far  its  use  with  established  Signal  Corps 
connections,  “They  .said,  Took,  we’ll  just  give  you  a 
IGBasc-T  connection  talking  TCP/HV  We  said, 
"Great,* w  retells  Utridicld.  Bur  down  the  road,  lit* 
tlefickfs  designers  still  sec  an  opportunity  -pardeu- 
larlv  if  trie  system  is  brought  to  commercialisation. 
They  want  to  add  RF  wireless  capability  in  future  k 
e radons  of  die  design.  Littlefield  sees  RF  sarellire 
links,  such  as  those  based  on  the  Immarsar  birds,  as 
the  most  likely  scenario  tor  this.  Dual-mode  cell¬ 
phone  technology  is  also  feasible,  he  says. 

Tuzla -tested 

The  MUSTPAO 1  prototype  was  shipped  in  July  of 
199 6,  but  its  first  real-world  trial  came  a  month  later 
when  Macedonia  rook  rhe  system  to  the  2I2lh 
MASH  ar  { lamp  Bedrock  in  Tuzla,  Bosnia.  Con¬ 
cerns  abour  reliability  were  quickly  put  to  rest  when 
Macedonia  wenr  ro  unload  the  system  after  it  took  a 
truck  ride  down  a  seven  mile  road  called  Crater  Al¬ 
ley.  To  his  dismay,  Macedonia  found  rhe  system  had 
bounced  off  its  palette  and  was  lying  in  a  corner  of 
die  back  of  the  truck.  “It  had  the  crap  kicked  out  of 
it,”  he  unabashedly  declares. 

Bur  the  unit  didn’t  M.  Once  set  up  at  MASH 
headquarters,  the  system  was  fired  up  and  linked  to 
the  Army's  local  I0base-T  Ethernet  network.  Data 
was  transmitted  back  through  a  microwave  link  io  a 
base  miles  away  and  then  fed  through  a  Ku  band 


Weighing  ubm!  85 
potctuh\  the  MUST- 
PAC-1  includes  an 
vh  ras*mMi  system  ,  a 
3-D  paddle,  and  a 
large  LCD  screen. 

A  Silicon  Crraplncs 
wv  rkst&tion-rtass 
Indy  computer — tinth 
camera — completes 
the  picture. 
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satellite  link  to  another 
M USTPAC  unit  in  Land- 
stuhf,  Germany  Thar  held 
test  proved  dearly  that  data 
could  be  collected  and  trans¬ 
mitted  by  personnel  inexpe¬ 
rienced  with  ultrasound 
technology. 

khl  initially  rhoughr  I  was  going  have  to  do  all  the 
scans,"  recalls  Macedonia.  “But  when  I  got  there, 
everyone  asked  to  try  it.  Eventually,  we  had  a  medic, 
a  Russian  nurse,  and  even  a  chaplain  try  it  our.”  I  he 
ability  of  diagnosticians  to  use  die  virtual  ultrasound 
probe  also  exceeded  expectations,  'dr  became  very 
dear  it  didn't  matter  who  was  doing  a  scan  "  re¬ 
counts  Macedonia. 

Saint  of  the  results  of  die  trip  were  completely 
unexpected.  “We  went  into  Bosnia  thinking  we  were 
taking  in  a  3-1.)  ultrasound  machine  and  came  out  of 
die  experience  realizing  we  had  what  could  he  called 
a  portable  telemedicine  system — that  at  its  core  had 
ultrasound^  avows  Macedonia. 

Since  die  system  does  frame  capture,  users  at  the 
MASH  quickly  began  experimenting  wirh  CCD  cam¬ 
eras  and  other  instruments  to  capture  eye  exams,  ear 
exams,  and  even  dental  exams.  They  were  anxious  to 
send  them  in  a  store  and -forward  fashion  back  to  the 
rear.  "The  people  at  the  MASH  taught  ns  that  it's  re¬ 


Users  with  tittle 
ultrasound  training 
can  collect  dam  fvr 
remote  analysis  ttsittfj 

the  MUSTPAC-l 

backpack  telemedicine 
system* 


ally  a  portable  rcienicxiicinc  system  that  allows  you  to 
examine  a  patient  from  t  Inc  top  of  their  head  ro  bottom 
of  their  teem  jocularly  notes  Macedonia. 


A  leading  designer  and  manufacturer  of  keyboards  and 
keypads  for  portable  input  products.,  our  technologies  include: 


•  Fiat  Pane!  /  Membrane 

*  Full-Travel  i  Notebook 

♦  Hand-Held  Products 

*  Complete  Assemblies 


•  Mid-Travel  i  Elastomer 

•  Full  Design  Services 

•  Models  £  Prototyp  es 

•  Complex  Control  Panels 


Call  (800)444-5923 

ADVANCED  INPUT  DEVICES 
West  2S0  AID  Drive 
Cceur  d'Alene.  ID  83814 
reieshons:  208-765^303 
Fax:  2G8-772-7613 
httpy>iwftiadtfarccd-lnput.ccrn 
E-nail:  saies-S a3vanced*hx:.ocm 
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A  lighter  future 

At  die  moment,  the  M USTPAC i  design  uvsm  is  t<>- 
c  using  on  reducing  system  weight  and  co st;  a  new  - 
generation  system  i.s  in  die  works.  Key  to  achieving 
the  goal  will  be  a  migration  to  a  lighter  and  more 
compact  Pentium- based  platform  that  will  include 
keyboard,  display  and  processor  in  a  single  mm. 
~Thst  should  eliminarc  about  10  pounds,”  believes 
I  irrieheki.  ‘TCI.I  eliminate  a  bunch  of  cable  and  sep 


arable  interconnect  problems  as  wdlT 

”  fhe  development's  a  continuous  evolution,''  con¬ 
cludes  Macedonia.  ~i  don't  see  an  end  to  die  Ml“ST- 


PAC-1  project  until  we  create  a  handheld  nor  a 
backpack — that  allows  me  to  have  a  look  at  people's 
bodies  and  figure  out  what's  going  mi  inside/"  ^ 


For  more  Information.*. 


Batteite/FaclUc  SKicon  Graphics 

Northwest  National  Mountain  View’.  GA  ; 

laboratory  {408}  960-1 980 

Richland,  WA  Circle  157 

(509)375-3776 

Circle  155  The  Fraunhofer  Center 

for  Research  In 

Hitachi  Medical  Computer  Graphics 

Systems  Providence,  R! 

Tanytown.  NY  .  (401}  454-7490 

524-971 1  Circle  158 

Circle  156 
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Backpack  remote  medicine  proves  its  worth 


A  portable  telemedicine  system  provid¬ 
ing  ultrasound  facilities  for  use  in 
field  hospitals  has  undergone  suc¬ 
cessful  trials  with  the  US  Army  in  Bosnia, 
Pacific  Northwest  Laboratory  (PNL)  devel¬ 
oped  the  Medical  UltraSound,  Three- 
dimensional  and  Portable  with  Advanced 
Communication  (MUSTPAC)  system 
under  a  rapid-prototyping  contract  from,  the 
US  Defense  Advanced  Research  Projects 
Agency  The  design  exploits  existing  technol¬ 
ogy  to  make  referral  and  diagnosis  easier  in 
the  field,  leading  to  improved  patient  care 
and  a  reduction  in  the  number  of  unneces¬ 
sary  evacuations. 

PNL  say's  that  MUSTPAC  is  unique  in 
that  it  is  effective,  requires  very*  little  training., 
docs  not  need  a  highly  skilled  operator  at  the 
patients  side-  and  operates  well  even  in  a 
low-bandwidth  store -and -forward  file- 
transfer  mode.  The  laboratory  is  developing 
a  pnxiuction-standard  version  that  is  lighter, 
more  compact,  and  uses  different  computer 
hardware, 

MUSTPAC  consists  of  a  field  unit 
installed  in  a  backpack,  allowing  it  to  be 
operated  as  far  forward  as  battalion  aid  sta¬ 
tions,  together  with  a  separate  virtual  ultra¬ 
sound  probe  and  high-resolution  color 
monitor.  In  its  prototype  form,  the  backpack 
contains  a  'three-dimensional  [3-D]  paddle 
electromechanical  scanner;  Silicon  Graphics 
Presenter  hat-panel  display;  Hitachi  EUR- 
905  ultrasound  machine;  Silicon  Graphics 
Indy  computer:  teleconferencing  video  cam¬ 
era;  and  keyboard  with  integral  touchpad. 
The  separate  Immersion  Probe  —  a  virtual 
ultrasound  probe  —  provides  expert  diag¬ 
nosticians  with  a  familiar  interface  that  Is 
easy-*  to  learn  and  to  use. 

The  system  permits  on-the-spot  visualiza¬ 
tion  of  internal  bleeding,  damage  to  solid  ; 
organs,  and  penetrating  injuries.  It  operates  j 


by  scanning  a  fairly  large  volume  of  the 
patient  at  one  rime,  so  that  diagnosis  during 
the  scan  is  not  required.  The  operator  places 
the  scanner  on  the  patient’s  abdomen  for 
about  10%  while  the  system  acquires  a  large 
volume  of  3-D  data.  Tills  requires  minimal 
training,  US  Marine  Corps  Capt  (now  Maj) 
Christian  Macedonia,  the  MUSTPAC  Prin¬ 
cipal  Medical  Investigator,  calked  a  Russian 
nurse  through  the  process,  via  an  interpreter, 
in  about  Strain  during  trials  in  Bosnia. 

The  equipment  also  provides  a  remote 
diagnostician  with  a  familiar  interface.  The 
‘virtual  ultrasound  probe’  consists  of  a  modl- 
ied  Immersion  Probe  or  an  equivalent  6-D 
sensing  capability  (3-D  position,  2-D  tilt, 
and  rotation).  IHeinVtVh  visualization  soft¬ 
ware  developed  by  the  Fraunhofer  Center 
for  Research  in  Computer  Graphics  uses 
probe  position  and  orientation  inputs  to 


Consulting  physicians  on  another  continent  can  use 
MUSTFAtfs  virtual-reality  technology  to  obtain  2-0 
diagnostic  views  tram  3-0  ultrasound  data.  An 
Immersion  Probe  provides  high-resolution  pictures 
of  scans  taken  elsewhere. 


*rcslice  die  3-D  ultrasound  data  along  arbi¬ 
trary  cutting  planes.  The  screen  view  is 
updated  in  real  time  (5-!  0  rimes  a  second)* 
so  that  diagnosticians  can  work  with 
MUSTPAC  3-D  scans  in  the  same  way  that 
they  would  with  a  real  patient  and  a  conven¬ 
tional  2-D  ultrasound  svseem. 

PNL  delivered  die  MUSTPAC- 1.  proto- 
1  type  to  the  US  Army  Medical  Research  and 
Materiel  Commands  Medical  Advanced 
Technology  Management.  Office  in  July  last 
year.  Following  a  series  of  pre-deployment 
evaluations  supervised  by  the  Center  for 
Total  Access  at  Fort  Gordon,  it  was  shipped 
to  Landstuhl  in  Germany.  In  early  August 
the  equipment  began  a  month-long  deploy¬ 
ment  with  the  212th  Mobile  Army  Surgical 
Hospital  at  Camp  Bedrock  in  TurJa,  Bosnia. 
It  transmitted  data  through  a  microwave  link 
*  to  Eagle  Base,  1 2 km  away,  and  then  fed 
j  through  a  Ku-band  satellite  link  to  another 
MUSTPAC  unit  in  Landstuhl.  Four  other 
receiving  sites  were  situated  In  the  US. 

The  MUSTPAC*  I  prototype  weighs  just 
under  40kg.  The  production-standard  ver¬ 
sion  now  under  development  will  be  more 
compact,  weigh  about  27kg,  and  incorporate 
a  laptop  computer  in  place  of  the  Indy  unit. 
The  equipment  may  then  enter  limited 
production  under  a  govern rn cn f Industry 
partnership.  Potential  users  include  rapid- 
reaction  forces  and  naval  unites. 

In  the  latter  ease,  MUSTPAC  could  assist 
in  detecting  gallstones,  diagnosing  kidney 
disease,  and  meeting  die  need  for  gynecolog¬ 
ical  care  dial  has  resulted  from  the  deploy¬ 
ment  of  women  aboard  warships.  The 
system  could  also  form  an  adjunct  to  image- 
directed  surgery-  for  immediate  treatment. 
MUSTPAC  could  show  the  position  of  both 
injuries  and  surgical  instruments,  supporting 
I  tasks,  such  as  the  injection  of  glue  to  sritefe 
i  internal  wounds.  MB 


The  MUSTPAO-1  prototype 
consists  of  a  virtual 
ultrasound  probe  (extreme 
left  of  photograph),  high- 
resolution  color  monitor 
(next  to  the  probe),  and 
a  backpack  (center) 
containing  the  other 
elements.  These  ccmprise 
an  electromechanical 
scanner  (extreme  right), 
flat-panel  display  (next 
to  it),  ultrasound  machine 
and  its  associated 
computer  (both  in 
backpack], 

teleconferencing  camera 
(on  backpack),  and 
keyboard  with  integral 
touchpad  (foreground). 


JAM'S  !NTS*NA7IONAv  D-H-NSr.  K^VIJ-W 


1-2 


APPENDIX  J 


APPENDIX  J:  “3-D  Ultrasound  for  Physiological  Monitoring” ,  presentation 
slide  set  from  the  DARPA  Ultrasound  Workshop,  Lansdowne,  VA, 
February  11-13, 1998. 
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Real  Time  3-D  Ultrasound 


Diagnostic  Screen  (human  liver) 


Familiar  diagnostic  interface 
Prototype  units  fielded  in  August  1 996 


Field  Trial _  Results 

ijglggL  .  Focused  on  usability  in  field  environment 


•Arrival  in  Tuzla  -  bounced  upside  down  on  Crater  Alley 


.  .  MUSTPAC-1  Principles  of 

Application  Targets  Operation 


TelelnViVo  software  (Fraunhofer  CRCG)  _  .  .  .  ,  . ,  „  „ 

'  Data  capture,  transfer,  and  Ul  software 

Data  capture,  transfer,  and  Ul  software  ....  . .  ri~[.  . 

Hitachi  905  ultrasound  engine 

Hitachi  905  ultrasound  engine 


Prototype  System  --  October  1997 


3-D  Dataset  Compression  -Summary 


7.58  MB  lossless  1 .26  MB,  JPEG 

(12.6  MB  raw) 

At  56  KB:  18  minutes  3  minutes 
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APPENDIX  K:  “MUSTPAC  3-D  Ultrasound  Telemedicine  -  The  MUSTPAC™ 
System”,  Web  site  http://aims.pnl.gov:2080/MUSTPAC-2/lndex.html,  as  of 
October  26,  1999. 
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3-D  Ultrasound 
Telemedicine  - 

The  MUSTPAC™ 
System 


Summary 

MUSTPAC™  (Medical  UltraSound,  Three-dimensional  and  Portable,  with  Advanced  Communications)  is 
a  telemedicine  system  based  on  3-D  ultrasound  data  acquisition. 

MUSTPAC™  provides  the  unique  capability  of  allowing  an  effective  ultrasound  examination  to  be 
performed  remotely  —  with  no  diagnostic  skills  needed  by  the  operator  at  the  patient’s  location,  and  no  need 
for  a  dedicated  videoconference  link. 

The  MUSTPAC™  technology  is  being  developed  at  the  Pacific  Northwest  National  Laboratory  in  support 
of  the  Advanced  Biomedical  Technologies  program  at  the  Defense  Advanced  Research  Projects  Agency 
tDARPA) 

Currently,  the  MUSTPAC-2  system  is  an  investigational  device  being  evaluated  in  both  clinical  and  field 
environments. 


Highlights  as  of  May  1998  include: 

•  A  ruggedized  MUSTPAC-2  configuration  accompanying  the  Everest  Extreme  Expedition  to 

MUSTPAC-2  system  is  ^e  Camp^  (ekval^nr^W^  ^ 

Everest  Extreme 
Expedition  to  collect 
scientific  data  on 
physiological 
adaptations  to  high 
altitude.  The  system  on 
Everest  incorporates  a 
Fieldworks  FW7666P 
rugged  laptop  and  an 
Ausonics  Impact  VFI 
portable  ultrasound 
scanner  with  Doppler 
capability.  This  system 


■(Bigger  picture) 


K-2 


is  supported  by  MU  S  TP  AC-2  diagnostic  workstation  at  Walter  Reed  Army  Medical  Center 

diagnostic  workstations  Telemedicine. 


located  at  Walter  Reed 
Army  Medical  Center 
and  Yale  University. 


A  standard  MUSTPAC-  musjpac^  system  in  Radiology  Department  of  National  Naval  Medical  Center. 

sys  em  as  een  (Shown  in  evaluation  configuration  with  ATL  HDI  ultrasound  scanner.) 
installed  at  the  National  appjMpr  ™ 

Naval  Medical  Center  ^  jj 


For  more  information... 


Several  detailed  documents  are  available  describing  the  current  MUSTPAC-2  system  and  its  predecessor 
MUSTPAC-1: 

•  System  design  and  experiences  of  the  MUSTPAC™  family  of  systems  are  described  in  a  7 -page 
paper  titled  "MUSTPAC™  3-D  Ultrasound  Telemedicine  /  Telepresence  System”,  presented  at  the 
1998  IEEE  International  Ultrasonics  Symposium,  Sendai,  Japan  (Oct.6-8).  This  paper  is  available 
as  a  Microsoft  Word  file. 

•  A  two-page  summary  handout,  from  the  DOE  Biomedical  Technologies  Exposition,  March  21, 
1999,  is  available  as  a  Microsoft  Word  file. 


•  A  large  photomontage  marketing  graphic  is  available  as  a  JPEG  file. 

•  System  design  and  field  experiences  of  the  MUSTPAC- 1  are  described  in  a  5-page  report  titled 
"MUSTPAC- 1:  3-D  Ultrasound  Telemedicine  Tool  for  Deployment  Situations  in  Bosnia  and  the 
European  Theater".  This  report  is  available  in  HTML  and  with  higher  quality  graphics  as  a 
Microsoft  Word  file. 

•  An  earlier  description  of  the  MUSTPAC- 1,  dated  December  1996,  is  also  available. 

•  A  December  1 997  briefing  presentation,  describing  the  MUSTPAC-2  system  with  application  to 
civilian  health  care,  is  available  as  a  Microsoft  PowerPoint  file. 

•  A  February  1998  presentation  given  at  the  DARPA  ultrasonics  workshop  also  is  available  as  a 
PowerPoint  File. 


APPENDIX  L 


APPENDIX  L:  “MUSTPAC™  3-D  Ultrasound  Telemedicine  /  Telepresence 
System ”,  1998  IEEE  International  Ultrasonics  Symposium,  Sendai,  Japan, 
October  5-8, 1998. 


Reprinted  with  permission. 
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Abstract  —  MUSTPAC™  is  a  family  of  ultrasound  tele¬ 
medicine  systems  based  on  the  use  of  3-D  (volumetric) 
data  acquisition.  Diagnostically  useful  scans  can  be  taken 
by  an  operator  with  limited  training  and  no  ultrasound 
diagnostic  skills,  using  a  simple  “point-and-shoot” 
procedure.  For  diagnosis,  volumetric  datasets  are 
transmitted  to  a  remote  radiologist,  who  uses  a  ‘Virtual 
ultrasound  probe”  to  reslice  the  datasets  along  arbitrary 
planes,  using  familiar  hand  movements  and  seeing  2-D 
(planar)  displays  similar  to  those  of  a  conventional  real¬ 
time  hands-on  examination.  Prototype  MUSTPAC™ 
systems  have  operated  in  a  military  field  hospital  and  on  a 
Mt.  Everest  climbing  expedition,  and  are  currently  being 
evaluated  for  clinical  application  in  rural  and  remote 
environments. 

Introduction 

Health-policy  experts  have  become  increasingly 
concerned  about  the  uneven  distribution  of  physicians. 
The  lack  of  widespread  availability  of  expertise  in 
diagnostic  ultrasound  has  been  a  particular  problem, 
causing  some  to  advocate  the  creation  of  ultrasound 
“centres  of  excellence”.[l]  Recently,  a  number  of  groups 
have  pioneered  realtime  remote  ultrasound  examination 
and  have  shown  the  value  of  teleconsultations  for 
ultrasound  technicians  in  rural  areas.[2,3, 4] 

However,  conventional  realtime  2-D  (two-dimensional) 
ultrasound  imaging  has  the  significant  drawback  that  a 
highly  skilled  operator  must  be  physically  present  at  the 
patient’s  location.  This  is  because  conventional  2-D 
ultrasound  imaging  uses  a  hands-on  interactive  procedure 
that  requires  the  operator  to  make  diagnostic  decisions 
simply  in  order  to  position  the  image  acquisition  probe  at 
the  correct  location  and  orientation. 

For  example,  to  allow  a  diagnosis  of  gallstones  using 
conventional  2-D  ultrasound,  the  operator  must 
interactively  manipulate  the  image  acquisition  probe  to 


locate  the  gall  bladder,  image  the  bile  duct  at  the  correct 
angle  to  measure  its  diameter,  and  finally  locate  the  stones 
within  the  bladder.  A  positioning  error  of  only  two  or  three 
millimeters,  relative  to  the  patient’s  internal  anatomy,  can 
make  the  difference  between  diagnostic  images  and  useless 
ones.  This  need  for  precision  pointing  introduces  some 
difficulties  in  using  conventional  2-D  ultrasound  in  a 
telemedicine  setting,  where  the  diagnostic  expert  does  not 
have  direct  control  over  the  probe  positioning. 

In  contrast,  using  3-D  (three-dimensional)  ultrasound 
potentially  allows  diagnostically  useful  scans  to  be  taken 
by  an  operator  with  limited  training,  no  diagnostic  skills, 
and  no  real-time  expert  assistance.  This  is  accomplished 
by  having  the  system  scan  a  fairly  large  volume  of  the 
subject’s  anatomy  at  one  time,  without  interpretation,  so 
that  the  operator  can  use  a  simple  “point-and-shoot” 
strategy  for  data  acquisition. 

For  example,  to  scan  for  gallstones  using  3-D  ultrasound, 
the  operator  has  to  know  only  enough  anatomy  to  scan  a 
volume  that  includes  the  gall  bladder.  Measuring  the  bile 
duct  and  locating  individual  stones  is  still  required,  but  this 
analysis  and  diagnosis  can  be  done  later  by  an  ultrasound 
expert  located  elsewhere. 

Beginning  in  1996,  a  series  of  prototype  3-D  ultrasound 
telemedicine  systems  has  been  developed  to  demonstrate 
and  evaluate  this  telemedicine  concept.  These  systems, 
called  MUSTPAC™  (Medical  UltraSound,  Three 
dimensional.  Portable,  with  Advanced  Communications), 
have  been  tested  in  a  variety  of  environments,  including  a 
U.S.  Army  telemedicine  network  in  Europe[5],  a  Mt. 
Everest  climbing  expedition,  and  several  medical 
institutions  in  the  U.S.  Clinical  studies  are  currently 
underway  to  evaluate  MUSTPAC™  for  routine  application 
in  rural  and  remote  settings. 

This  report  discusses  the  design,  implementation,  and 
use  of  several  MUSTPAC™  systems. 


t  U.S.  Army  Medical  Corps,  Georgetown  University  Medical  Center,  3800  Reservoir  Road,  Washington,  DC  20007 
*  Fraunhofer  Center  for  Research  in  Computer  Graphics  (CRCG),  321  S.  Main  Street,  Providence,  RI  02903 
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Overview 

MUSTPAC™  is  an  ultrasound  medical  imaging  system 
that  can  scan  patients  to  generate  3-D  volumetric  digital 
datasets,  interactively  generate  3-D  and  2-D  images  for  use 
by  diagnosticians,  and  optionally  transfer  datasets  over 
standard  communication  links  to  facilitate  remote  diagnosis 
and  consultation.  It  is  designed  to  work  in  a  telemedicine 
framework,  enabling  diagnostically  useful  ultrasound  scans 
to  be  taken  by  an  operator  with  no  diagnostic  skills, 
modest  training,  and  no  online  connection  to  an  expert. 

Typically  a  MUSTPAC™  system  is  used  as  follows.  First, 
the  patient  is  scanned  by  placing  an  ultrasound  probe  on 
the  patient  and  mechanically  sweeping  it  across  his/her  skin 
over  the  area  of  interest  (Figure  1).  During  the  scan,  the 
system  records  ultrasound  data  from  a  sizable  3D  volume 
of  the  patient’s  anatomy,  producing  a  3D  volumetric  dataset 
of  ultrasound  reflectivity.  The  scanning  process  requires  no 
interpretation  of  the  ultrasound  images,  other  than  possibly 
to  confirm  that  the  intended  anatomy  is  covered. 


Figure  1.  Acquiring  a  3-D  abdominal  scan  using  a 
linear  scanning  device  and  conventional  ultrasound 
transducer. 


Scans  in  the  form  of  3D  volumetric  datasets  are  then 
transmitted  over  any  standard  digital  network  to  a  qualified 
diagnostician. 

Finally,  a  diagnostician  interprets  each  3-D  scan  using  a 
Virtual  Ultrasound  Probe  that  simulates  a  conventional 
real-time  hands-on  examination  procedure.  This  allows  the 
diagnostician  to  display  arbitrary  2D  slices  from  the  3D 
dataset  simply  by  moving  the  probe  as  if  he/she  were 
interactively  examining  the  patient.  The  Virtual  Ultrasound 
Probe  and  corresponding  screen  displays  are  very  natural  to 
diagnosticians,  leading  to  rapid  acceptance  and 
productivity  (Figures  2-4). 


Figure  2.  Interpreting  a  3-D  scan  using  the  Virtual 
Ultrasound  Probe  at  a  diagnostic  workstation. 


Figure  3.  Closeup  of  the  Virtual  Ultrasound  Probe 
and  diagnostic  workstation  display.  Image  on  the 
screen  is  of  a  plastic  phantom. 


Figure  4.  Actual  diagnostic  display  (human  liver  with 
clotting  in  major  vessels). 
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Principles  of  operation 

All  versions  of  the  MUSTPAC™  system  developed  to 
date  have  used  a  similar  high  level  design  (Figure  5). 
There  are  two  primary  operations  —  scanning  and 
visualization  —  typically  separated  by  a  data  transmission 
step. 


Volume 

Data  File  wMxfW’k 


Visualization  Software 

3-D  Volumetric  -* 
2-D  Slices 
(arbitrary  reslicing) 


T el  ecom  muni  cat  ion : 
file  transfer 

interactive  collaboration 


Figure  5.  Major  aspects  of  the  MUSTPAC™ 
architecture.  In  this  diagram,  scanning  and 
visualization  are  shown  on  the  same  system.  In 
practice,  these  two  functions  would  normally  be  done 
on  two  systems. 


In  freehand  scanning,  the  transducer  is  hand-held  and 
moved  by  the  operator.  A  6-degree-of-freedom  sensor 
attached  to  the  ultrasound  transducer  records  its  position 
and  orientation.  Software  in  the  MUSTPAC™  system  then 
reconstructs  a  dense  3-D  volumetric  dataset  by 
interpolating  as  necessary  between  the  somewhat  random 
planes  of  acquired  image  data.  This  procedure  generates 
somewhat  lower  quality  datasets,  primarily  due  to 
inaccuracies  in  sensing  the  probe  position.  The  quality  of 
freehand  datasets  depends  to  a  considerable  degree  on 
properties  of  the  position  sensor  and  its  interaction  with  the 
operating  environment.  The  MUSTPAC™  development 
group  is  currently  evaluating  the  tradeoffs  between  free- 
space  magnetic  sensors  (such  as  the  Ascension1  pcBIRD™) 
and  mechanical  articulated  arms  (such  as  the  Immersion2 
MicroScribe-3D™). 

Scans  are  usually  taken  at  15  fps  (frames  per  second)  at  a 
scanning  rate  of  1  cm/sec,  using  a  3.5  MHz  or  5  MHz 
convex-face  sector-scan  probe.  With  typical  penetration 
depths,  this  results  in  scanning  a  region  4-6  cm  wide  at  the 
skin  surface  and  20  cm  wide  at  15  cm  deep,  with  a  voxel 
(volume  element)  size  of  roughly  0.7mm  on  each  axis.  The 
resulting  3D  rectangular  dataset  contains  about  13  million 
voxels. 

Visualization  for  Analysis  and  Diagnosis 


Scanning 

Volumetric  scanning  in  MUSTPAC™  is  done  using  the 
well-known  approach  [6]  of  mechanically  moving  the 
transducer  probe  of  a  conventional  off-the-shelf  ultrasound 
machine.  As  the  probe  moves,  the  ultrasound  machine 
generates  a  sequence  of  video  images,  called  B-scans, 
showing  2-D  slices  of  the  subject  under  the  probe.  A 
computer  digitizes  the  sequence  of  B-scan  images  and 
corresponding  spatial  positions  of  the  transducer  probe, 
then  assembles  the  B-scans  to  form  a  3-D  volumetric 
dataset  that  represents  a  sort  of  “snapshot”  of  the  subject’s 
anatomy.  Anatomical  movement  is  not  captured,  except  in 
the  form  of  motion  artifacts  that  the  diagnostician  must 
learn  to  ignore. 

MUSTPAC™  supports  two  types  of  volumetric  scanning: 
linear  and  freehand.  In  linear  scanning,  a  hand-held  motor- 
driven  device  is  used  to  move  the  transducer  in  a  straight 
line,  at  a  constant  speed,  keeping  the  transducer  oriented 
perpendicular  to  the  direction  of  movement.  This  generates 
a  set  of  parallel  registered  images  that  naturally  assemble  to 
form  a  dense  rectangular  3-D  dataset.  Linear  scanning  is 
fairly  restrictive  in  what  anatomy  it  can  be  used  with. 
However,  the  technique  generates  very  high  quality 
datasets  because  tightly  controlling  the  transducer  position 
essentially  eliminates  the  possibility  of  position  error  and 
gaps  in  the  image  data. 


The  primary  function  of  visualization  in  MUSTPAC™  is 
to  allow  experienced  sonologists  to  make  diagnostic 
decisions.  Thus,  the  emphasis  is  on  providing  to  those 
diagnosticians  a  user  interface  that  is  familiar,  easy  to 
learn,  and  productive. 

Screen  displays.  The  primary  visualization  tool  in 
MUSTPAC™  is  TelelnViVo™3.  TeleInViVo™can  generate 
several  kinds  of  images: 

•  2-D  slices  at  arbitrary  positions  and  orientations. 

•  3-D  volumes  using  "maximum  intensity  projection" 
(MIP)  and  similar  techniques  for  combining  the 
values  of  dataset  volume  elements  (voxels). 

•  3-D  surfaces  defined  by  threshold  values  and  rendered 
using  one  of  several  shading  techniques 

In  MUSTPAC™,  the  most  common  practice  is  to 
combine  a  large  2-D  slice  image  with  a  small  3-D 
volumetric  image  (Figure  4).  The  2-D  slice  shows  detail 
that  is  diagnostically  useful,  while  the  3-D  volumetric 
image  provides  context  by  showing  location  of  the  2-D 
slice  within  the  volume. 

Virtual  Ultrasound  Probe .  To  provide  diagnosticians 
with  a  familiar,  convenient  user  interface,  the  MUSTPAC™ 


1  Ascension  Technology  Corp.,  Burlington,  VT, 

http://www.ascension-tech.com 

2  Immersion  Corp,  San  Jose,  CA,  http://www.immerse.com 

3  Fraunhofer  CRCG,  Providence,  RI,  http://www.crcg.edu 
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version  of  TelelnViVo™  has  been  extended  by  the  addition 
of  a  Virtual  Ultrasound  Probe  (Figure  3).  This  is  a  hand¬ 
held  6-D  input  device  combined  with  software  to  control  2- 
D  slicing  of  volumetric  datasets  in  the  same  way  that  a  real 
ultrasound  probe  controls  2-D  imaging  of  patients.  That  is, 
the  Virtual  Ultrasound  Probe  looks,  feels,  and  acts  like  a 
real  ultrasound  probe  in  that  the  on-screen  image  is 
constantly  updated  in  real  time  (typically  5-10  times  per 
second)  to  reflect  the  position  and  orientation  of  the  probe. 
(Again,  this  real  time  updating  does  not  reflect  anatomical 
movement,  which  is  not  captured  in  the  datasets.) 


•  Hitachi  EUB-905™  ultrasound  machine  (in  backpack, 
top  section,  with  cord) 

•  Silicon  Graphics  Indy™  computer  (in  backpack 
bottom  section) 

•  Teleconferencing  camera  (on  backpack,  top  left) 

•  Keyboard  with  integral  touchpad. 

•  High-resolution  color  monitor. 

•  Virtual  Ultrasound  Probe 

•  TelelnViVo™  visualization  software. 

•  Other  custom  data  acquisition  and  control  software. 


Data  Transmission 

The  MUSTPAC™  system  provides  two  major  methods  of 
data  communication:  batch  mode  file  transmission  and 
TelelnViVo™  incremental  transmission. 

In  batch  mode  file  transmission,  3-D  datasets  and 
associated  files  are  transferred  using  the  Internet  standard 
"ftp”  protocol.  If  bandwidth  is  limited,  then  the  3-D 
datasets  are  compressed  before  transmission,  using 
standard  JPEG4  methods.  On  typical  datasets,  this  achieves 
approximately  10X  compression  without  visible  effect  on 
the  images. 

Incremental  communication  capabilities  are  also  built 
into  the  TelelnViVo™  visualization  program.  These 
capabilities  allow  all  or  part  of  the  full  dataset  to  be 
transferred  quickly  at  reduced  resolution,  while  the  user 
interactively  evaluates  the  images.  This  feature  has  not 
been  used  extensively  to  date,  but  is  expected  to  become 
more  frequent  as  MUSTPAC™  systems  are  incorporated 
into  consulting  practices.  Typical  use  is  envisioned  to  be 
transfer  of  the  entire  dataset  with  resolution  reduced  by  4X 
per  axis  (64X  overall),  followed  by  transmission  of  an 
identified  region  of  interest  at  the  full  resolution  of  the 
dataset. 


IMPLEMENTATIONS 

MUSTPAC- 1 

In  the  summer  of  1996,  the  first  prototype  system  was 
developed  for  testing  by  the  U.S.  Army  under  field 
conditions.  This  system,  called  the  MUSTPAC- 1,  was 
designed  to  demonstrate  an  ultrasound  telemedicine 
system  in  a  backpack. [7,8] 

System  components  and  packaging.  Major  components  of 
the  MUSTPAC-1  are  shown  in  Figure  5.  These 
components  include  (right-to-left): 

•  Battery-powered  linear  scanner  (at  extreme  right) 

•  Silicon  Graphics  Presenter™  flat  panel  display. 


4  Joint  Photographic  Experts  Group, 


ftp://ftp.uu.net/graphics/jpeg 


Figure  5.  MUSTPAC-1  system  components  as 
packaged  for  military  field  evaluation. 


Field  Experiences .  In  August,  1996,  the  MUSTPAC-1 
system  was  deployed  by  the  U.S.  Army  to  the  212th 
Mobile  Army  Surgical  Hospital  in  Tuzla,  Bosnia.  A 
second  MUSTPAC-1  placed  at  the  Army’s  Landstuhl 
Regional  Medical  Center  in  Germany  served  as  the  primary 
receiving  station. 

Scanning.  During  the  deployment,  a  total  of  72  scans 
were  performed.  Most  of  these  scans  were  taken  by 
operators  with  no  ultrasound  diagnostic  skills  and  minimal 
MUSTPAC-1  training  (typically  10  minutes).  No  formal 
evaluation  of  the  image  quality  was  performed;  informal 
evaluation  by  a  variety  of  experienced  ultrasound  users 
suggested  that  the  quality  of  the  scans  was  largely 
independent  of  an  operator’s  level  of  training  and  generally 
ranged  from  adequate  to  good. 

Communications.  Most  transfers  between  Bosnia  and 
Germany  were  performed  using  ftp  over  a  roughly  1 
megabit/second  geosynchronous  satellite  link  leased  by  the 
Army.  Despite  a  round-trip  packet  delay  of  580-600  ms, 
net  transfer  rates  of  roughly  50  Kbytes/second  were 
routinely  achieved.  Thus  typical  3-D  datasets  of  6-12 
Mbytes  required  only  a  few  minutes  to  transfer  even 
without  compression. 

As  exercises,  3-D  datasets  were  also  transferred  over  two 
slower  communication  links.  One  of  these  links  was 
between  Germany  and  Washington  DC  (USA)  at  56 
Kbits/sec  using  the  International  Maritime  Satellite  System 
(INMARSAT)  and  its  associated  telephone  system  ISDN 
link.  The  other  was  between  Bosnia  and  Germany  using 
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the  Army’s  Tactical  Packet  Network  (TACNET)  at  9.6 
Kbits/sec.  These  transfers  were  also  successful,  but  would 
have  been  too  slow  for  routine  use.  (Data  compression  was 
added  to  the  MUSTPAC-2  to  address  this  deficiency.) 

Diagnostic  Usability.  During  the  Bosnia  deployment  and 
in  the  6  months  after  its  completion,  the  MUSTPAC-1 
system  was  operated  by  approximately  20  experienced 
ultrasound  users.  Again,  informal  evaluation  suggested  that 
the  system  was  very  easily  learned.  One  striking 
observation  was  that  no  experienced  ultrasound  user 
required  more  than  5  minutes  practice  with  the  Virtual 
Ultrasound  Probe  to  begin  making  medical  interpretations 
of  3-D  scans  that  he/she  had  not  previously  seen. 

MUSTPAC-2 

MUSTPAC-2  is  actually  a  family  of  MUSTPAC™ 
systems  implemented  on  a  common  base  of  Intel  Pentium 
processors  using  the  Windows/NT  operating  system.  The 
principles  of  operation  are  similar  to  MUSTPAC-1. 
However,  MUSTPAC-2  incorporates  several  significant 
improvements  from  the  user’s  standpoint.  These  include: 

•  Simplified  and  more  robust  user  interface 

•  Wider  range  of  packaging  options  (e.g.  lightweight 
and  rugged  data  acquisition  system  versus  high 
performance  diagnostic  workstation) 

•  Increased  network  compatibility. 

•  Faster  transmission  and  reduced  data  volume  (through 
JPEG  data  compression). 

•  DICOM  v3.0  import/export  capability5. 

•  Freehand  scan  (research  quality  -  still  under 
development  for  routine  clinical  use). 

Several  configurations  of  MUSTPAC-2  systems  have 
been  developed  for  specific  purposes.  These  purposes 
include: 

•  Everest  Extreme  Expedition 

•  Hospital  clinical  evaluation 

•  Remote  clinic  consultation 

Everest  Extreme  Expedition .  An  unusually  rugged  portable 
configuration  of  MUSTPAC-2  (Figure  6)  accompanied  the 
Yale/NASA  telemedicine  team  on  the  May  1998  Everest 
Extreme  Expedition6  to  Base  Camp  (elev.  17,500  ft).  3-D 
ultrasound  datasets  were  transmitted  back  to  MUSTPAC™ 
diagnostic  workstations  at  Yale  University  (New  Haven, 
CT)  and  Walter  Reed  Army  Medical  Center  (Washington, 
DC)  for  interpretation.  In  addition,  the  MUSTPAC™ 
system  acquired  a  series  of  2-D  images  containing  spectral 
Doppler  data  characterizing  blood  flow  in  major  arteries,  as 


5  Merge  Technologies,  Milwaukee,  WI,  http://www.merge.com 

g 

http://www.explorers.org/newsfiles/e3.htmI 


part  of  a  scientific  study  of  climber  adaptations  to  high 
altitude. 


Figure  6.  MUSTPAC-2  Everest  Extreme  configuration. 
This  system  incorporated  a  166  MHz  FieldWorks  FW- 
7666P  computer7  (center),  Ausonics  Impact  VFI 
ultrasound  machine8  (upper  left),  MUSTPAC-1  battery 
powered  linear  scanner,  and  MUSTPAC-1  Virtual 
Ultrasound  Probe. 

Hospital  clinical  evaluation .  A  deskside  system  (Figure  7) 
equipped  for  both  scanning  and  visualization  has  been 
installed  at  the  National  Naval  Medical  Center  (NNMC)  in 
Bethesda,  MD.  This  system  is  being  evaluated  for  use  in  a 
clinical  setting,  prior  to  possible  shipboard  deployment. 


Figure  7.  MUSTPAC-2  clinical  evaluation  system  at 
NNMC.  This  unit  incorporates  a  333  MHz  Dell 
Dimension  XPS  computer9  (under  desk),  MUSTPAC-2 
Virtual  Ultrasound  Probe  based  on  the  Microscribe  3D 
articulated  arm10  (on  table),  MUSTPAC-2  linear 
scanner  (on  table),  and  NNMC’s  existing  ATL  HDI- 
3000  ultrasound  machine11  (console  and  cabinet  at  left). 


7  Fieldworks  Inc.,  Eden  Prairie,  MN,  http://www.field-works.com 

8 

Universal  Medical  Systems,  Bedford  Hills,  NY,  http://www.u-m-s.com 

Q 

Dell  Computer  Corporation,  Round  Rock,  TX,  http://www.deli.com 

10  Immersion  Corporation,  San  Jose,  CA,  http://www.immerse.com 

11  ATL  Ultrasound,  Bothell,  WA,  http://www.atl.com 
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Remote  clinic  consultation.  The  “lunchbox”  configuration 
(Figure  8)  focuses  on  data  acquisition,  with  limited 
capability  for  visualization.  This  demonstration/evaluation 
unit  represents  a  configuration  being  considered  for 
installation  in  a  network  of  telemedicine  clinics  in  rural 
northern  U.S. 


Figure  8.  This  portable  “lunchbox”  configuration 
incorporates  a  166  MHz  BSI  LCD  V8  computer12, 
MUSTPAC-2  linear  scanner  (not  shown),  MUSTPAC-2 
Virtual  Ultrasound  Probe,  and  any  of  several  existing 
ultrasound  machines  (also  not  shown). 


Discussion 

Experience  to  date  suggests  that  MUSTPAC™  is  a 
promising  approach  for  extending  the  application  of 
ultrasound  in  telemedicine.  Further  studies  are  needed, 
however,  to  determine  its  usefulness  in  clinical  practice.  In 
addition,  MUSTPAC™  has  a  number  of  recognized 
limitations  that  need  to  be  addressed.  These  include: 

•  No  Doppler  imaging.  At  present,  MUSTPAC™  has  no 
specific  capability  to  capture  or  visualize  Doppler 
information  in  3-D  datasets,  suitable  for  later  reslicing. 

•  No  anatomic  motion.  Conventional  real-time  2-D 
ultrasound  shows  anatomic  motion,  such  as  pulsing 
arteries,  as  a  moving  image  on  screen.  This  is 
diagnostically  useful.  However,  the  3-D  scans 
captured  by  MUSTPAC™  are  static  snapshots. 
Generally  speaking,  anatomic  motion  produces  image 
artifacts  that  the  diagnostician  must  leam  to  recognize 
and  ignore.  In  extreme  cases,  such  as  scanning  an 
active  fetus,  anatomic  movement  can  render  a  dataset 
unusable.  This  condition  must  be  recognized  by  the 
scanner  operator,  and  the  scan  repeated. 


•  Static  shadowing  artifacts .  With  conventional  realtime 
ultrasound,  the  image  is  produced  by  insonification 
from  the  current  transducer  position,  so  shadow  and 
posterior  enhancement  artifacts  always  look  radial  and 
shift  location  if  the  transducer  moves.  With 
MUSTPAC™,  these  artifacts  are  captured  statically  as 
they  appeared  during  data  acquisition.  When  the  3-D 
dataset  is  resliced  on  a  different  plane,  shadows  and 
enhancements  can  have  an  unusual  appearance. 
Again,  diagnosticians  must  leam  to  recognize  and  deal 
with  these  artifacts,  typically  by  reviewing  the  data  in 
the  original  plane  of  insonification  to  clarify  the 
interpretation. 

•  Data  quality  bounded  by  the  original  B-scans.  In  the 
conventional  clinical  setting,  skilled  ultrasound 
technicians  interactively  adjust  power,  gain,  TGC 
(time  gain  compensation),  patient  position,  and  other 
parameters  to  optimize  images  on  a  case-by-case  basis. 
At  present,  the  MUSTPAC™  system  cannot  make  such 
adjustments  by  itself  (since  it  has  no  direct  control 
over  the  ultrasound  machine),  nor  does  it  actively  help 
the  scanner  operator  to  make  them.  Carefully  chosen 
presets  help  to  alleviate  this  problem.  In  the  long  run, 
we  anticipate  that  automated  image  analysis  will 
enable  further  improvements. 

Summary 

3-D  ultrasound  data  acquisition  potentially  can  expand 
the  use  of  ultrasound  imaging  in  a  telemedicine  setting,  by 
allowing  an  operator  with  no  diagnostic  skills  to  collect 
high  quality  scans  that  can  be  interpreted  by  a  remote 
expert.  This  potential  is  illustrated  by  the  MUSTPAC™ 
series  of  ultrasound  telemedicine  systems.  Several 
prototype  MUSTPAC™  systems  have  been  developed  and 
evaluated  in  a  variety  of  operational  environments.  While 
the  MUSTPAC™  approach  is  promising,  significant  further 
work  remains  before  its  clinical  applicability  is  fully 
determined. 
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Disclaimers 

The  opinions  expressed  in  this  article  are  those  of  the 
primary  authors  and  do  not  necessarily  reflect  official 
policy  of  the  United  States  Department  of  Defense, 
Department  of  Energy,  Battelle,  or  Fraunhofer. 

Reference  to  any  specific  commercial  product,  process, 
or  service  by  trade  name,  trademark,  manufacturer,  or 
otherwise,  does  not  necessarily  constitute  or  imply  its 
endorsement,  recommendation,  or  favoring  by  the  authors 
or  their  associated  organizations. 

MUSTPAC™  is  a  trademark  of  Battelle  Memorial 
Institute.  TelelnViVo™  is  a  trademark  of  Fraunhofer 
CRCG.  All  other  trademarks  are  property  of  then- 
respective  owners. 
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Problem  Statement 


1.0 


Summary 


Mercy  Health  System  has  proposed  to  the  Department  of  Defense  to  conduct  a  study  showing  how  the  use 
of  telemedicine  devices  will  expedite  the  triage  and  treatment  process  in  emergency  situations  for  patients 
in  remote  areas,  thereby  improving  timeliness,  quality  of  care  and  patient  outcomes.  Data  generated  from 
this  study  will  be  compared  to  other  telemedicine  devices  with  the  goal  of  establishing  cost  effective 
telemedicine  devices  for  use  in  the  remote  and  austere  battlefield  environment  with  technology  transfer  to 
the  civilian  at-home  or  assisted-living  environments. 

Mercy  Health  System  proposes  to  include  the  MUSTPAC™  3-D  Ultrasound  System  in  its  study  to 
determine  if  sonographic  data  acquired  in  volumetric  fashion  can  be  used  by  a  physician  at  a  site  remote 
from  where  the  data  was  obtained  for  a  diagnosis  with  accuracy  equal  to  that  of  the  “gold  standard”  in- 
hospital  real-time  protocol. 

Project  Objective  Statement 

Deliver  one  MUSTPAC-3  prototype  five  months  after  contract  award  to  Battelle  Pacific  Northwest 
Division  (Battelle)  for  test  and  evaluation  by  Mercy  Health  System,  and  provide  assistance  for  three 
months  to  Mercy  Health  System  in  performing  that  evaluation,  for  $372,609.  MUSTPAC-3  will  consist 
of  engineering  improvements  that  incorporate  free-hand  scan  for  gray  scale  into  the  MUSTPAC-2  system 
delivered  under  DOD  Contract  DAMD17-94-C-4127.  Battelle  also  will  pursue  FDA  510(k)  approval  for 
the  MUSTPAC-3  prototype  using  data  resulting  from  the  Mercy  Health  System  study. 

Project  Benefits 

Mercy  Health  System  is  a  major  provider  of  the  medical  and  health  services  used  by  residents  of 
Delaware  County,  Pennsylvania.  The  proposed  collaborative  research  endeavor  will  generate  data 
relative  to  the  improvement  in  acceptance  and  delivery  of  telemedicine  applications  through  the 
development  of  systems,  policies,  protocols,  procedures,  and  educational  products  applicable  to  medical 
services  management  efforts  both  in  the  military  and  civilian  health  delivery  sectors.  Mercy  Health 
System  plans  to  use  the  MUSTPAC™  system  as  part  of  its  state-of-the-art  telemedicine  network  to 
improve  the  care  provided  in  both  emergency  and  non-emergency  environments. 

Mercy  plans  to  conduct  its  study  in  two  phases.  Phase  1  will  consist  of  an  in-house  clinical  trial  that  will 
compare  MUSTPAC-3  to  the  current  “Gold  Standard”  ultrasound  methodology.  If  Phase  1  successfully 
demonstrates  the  usefulness  of  the  MUSTPAC  system,  then  Phase  2  will  take  MUSTPAC-3  into  the 
remote  environment.  Mercy  will  extend  the  use  of  the  system  to  nursing  homes,  external  health  clinics 
and  ambulance  services.  Data  will  be  collected  at  the  remote  sites  and  transmitted  to  diagnostic  experts 
centrally  located  in  the  major  Mercy  facilities  for  interpretation  and  analysis. 
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Deliverables 


Deliverable 

Description 

MUSTPAC-3  Prototype,  1  each,  to 
support  Phase  1  of  the  Mercy  study 

MUSTPAC  system  with  free-hand  scan  capability: 

•  3-D  Free  hand  scanner  device  for  data  collection 

•  Virtual  Probe  for  data  interpretation 

•  Integrated  MUSTPAC  software 

System  Installation 

Install  MUSTPAC  -3  Prototype  on  a  Mercy  computer 
and  calibrate  the  system  for  use  with  a  Mercy  ultrasound 
instrument  in  a  clinical  environment 

Monthly  Project  Status  Report 

Monthly  status  report 

3.0  Scope 

Execution  of  this  project  is  dependent  on  support  from  two  sponsors:  Mercy  Health  System,  Bala 
Cynwyd,  Pennsylvania,  and  McKennan  Medical  System,  Sioux  Falls,  South  Dakota.  The  earlier  versions 
of  MUSTPAC™  were  developed  under  a  research  project  of  the  Defense  Advanced  Research  Projects 
Agency  (DARPA).  DARPA’s  research  objectives  have  been  met  and  the  project  has  moved  into  the 
technology  transfer  stage  which  is  not  being  funded  by  DARPA.  By  jointly  sponsoring  the  continuation 
of  MUSTPAC  research,  Mercy  Health  System  and  McKennan  Medical  System  will  benefit,  since  they 
share  a  common  desire  to  significantly  improve  the  quality  of  healthcare  provided  to  their  patients 
through  the  application  of  emerging  telemedicine  systems  like  MUSTPAC™. 

A  single  version  of  the  MUSTPAC-3  prototype  will  be  developed  for  both  sponsors  and  delivered  in  the 
quantities  specified  in  the  individual  project  proposals  of  each  sponsor.  By  sponsoring  the  continuation 
of  the  research  each  sponsor  will  receive  an  exclusive  use  license  for  its  respective  patient  care  region. 

Battelle  will  deliver  the  MUSTPAC-3  software,  the  3-D  free  hand  scanner  for  data  collection,  the  virtual 
probe  for  data  analysis,  and  the  computing  platform  to  run  these.  Mercy  Health  System  will  provide  the 
ultrasound  instrument  that  will  be  used  with  the  MUSTPAC-3  system.  Battelle  will  calibrate  the  Mercy 
MUSTPAC  system  for  use  with  the  Mercy  ultrasound  instrument.  Battelle  also  will  provide  user  training 
concurrently  with  MUSTPAC  installation  and  setup.  A  duplicate  MUSTPAC-3  system  will  be  placed  at 
Battelle  to  support  continued  development  and  to  serve  as  a  “hot  spare”  in  the  event  of  equipment  failure 
at  Mercy. 

Since  MUSTPAC  is  an  experimental  medical  device,  it  can  be  used  only  within  the  scope  of  a  medical 
research  protocol  approved  by  appropriate  IRBs  (Institutional  Review  Boards).  Mercy  is  responsible  for 
preparing  the  protocol  as  well  as  submitting  the  protocol  to  the  Mercy  IRB.  Federal  regulations  relating 
to  Battelle’s  government  contracts  require  that  the  protocol  also  be  approved  by  the  IRB  at  the  Pacific 
Northwest  National  Laboratory  (PNNL)  in  Richland,  WA.  Battelle  will  assist  Mercy  staff  in  the 
preparation  of  the  protocol,  to  facilitate  its  acceptance  by  both  IRBs.  During  the  clinical  trial  Battelle  will 
provide  telephonic  technical  support  to  the  Mercy  staff. 
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Battelle  will  prepare  and  submit  a  device  approval  application  to  the  U.S.  Food  and  Drug  Administration 
using  the  clinical  trial  data  collected  by  Mercy  and  McKennan  (FDA  510k  application). 

4.0  Background 

Introduction 

MUSTPAC™  is  a  family  of  ultrasound  telemedicine  systems  based  on  the  use  of  3-D  (volumetric)  data 
acquisition.  An  operator  with  limited  training  and  no  ultrasound  diagnostic  skills,  using  a  simple  “point- 
and-shoot”  procedure,  can  take  diagnostically  useful  scans.  For  diagnosis,  volumetric  datasets  are 
transmitted  to  a  remote  radiologist,  who  uses  a  “virtual  ultrasound  probe”  to  reslice  the  datasets  along 
arbitrary  planes,  using  familiar  hand  movements  and  seeing  2-D  (planar)  displays  similar  to  those  of  a 
conventional  real-time  hands-on  examination. 

Health-care  policy  experts  have  become  increasingly  concerned  about  the  uneven  distribution  of 
physicians.  The  lack  of  widespread  availability  of  expertise  in  diagnostic  ultrasound  has  been  a  particular 
problem.  Recently,  a  number  of  groups  have  pioneered  real-time  remote  ultrasound  examination  and 
have  shown  the  value  of  tele-consultations  for  ultrasound  technicians  in  rural  areas. 

However,  conventional  real-time  2-D  (two-dimensional)  ultrasound  imaging  has  the  significant  drawback 
that  a  highly  skilled  operator  must  be  physically  present  at  the  patient’s  location.  This  is  because 
conventional  2-D  ultrasound  imaging  uses  a  hands-on  interactive  procedure  that  requires  the  operator  to 
make  diagnostic  decisions  simply  in  order  to  position  the  image  acquisition  probe  at  the  correct  location 
and  orientation. 

For  example,  to  allow  a  diagnosis  of  gallstones  using  conventional  2-D  ultrasound,  the  operator  must 
interactively  manipulate  the  image  acquisition  probe  to  locate  the  gall  bladder,  image  the  bile  duct  at  the 
correct  angle  to  measure  its  diameter,  and  finally  locate  the  stones  within  the  bladder.  A  positioning  error 
of  only  two  or  three  millimeters,  relative  to  the  patient’s  internal  anatomy,  can  make  the  difference 
between  diagnostic  images  and  useless  ones.  This  need  for  precision  pointing  introduces  some 
difficulties  in  using  conventional  2-D  ultrasound  in  a  telemedicine  setting,  where  the  diagnostic  expert 
does  not  have  direct  control  over  the  probe  positioning. 

In  contrast,  using  3-D  (three-dimensional)  ultrasound  potentially  allows  diagnostically  useful  scans  to  be 
taken  by  an  operator  with  limited  training,  no  diagnostic  skills,  and  no  real-time  expert  assistance.  This  is 
accomplished  by  having  the  system  scan  a  fairly  large  volume  of  the  subject’s  anatomy  at  one  time, 
without  interpretation,  so  that  the  operator  can  use  a  simple  “point-and-shoot”  strategy  for  data 
acquisition. 

For  example,  to  scan  for  gallstones  using  3-D  ultrasound,  the  operator  has  to  know  only  enough  anatomy 
to  scan  a  volume  that  includes  the  gall  bladder.  Measuring  the  bile  duct  and  locating  individual  stones  is 
still  required,  but  this  analysis  and  diagnosis  can  be  done  later  by  an  ultrasound  expert  located  elsewhere. 

System  Overview 

MUSTPAC™  is  an  ultrasound  medical  imaging  system  that  can  scan  patients  to  generate  3-D  volumetric 
digital  datasets,  interactively  generate  3-D  and  2-D  images  for  use  by  diagnosticians,  and  optionally 
transfer  datasets  over  standard  communication  links  to  facilitate  remote  diagnosis  and  consultation.  It  is 
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designed  to  work  in  a  telemedicine  framework,  enabling  diagnostically  useful  ultrasound  scans  to  be 
taken  by  an  operator  with  no  diagnostic  skills,  modest  training,  and  no  online  connection  to  an  expert. 

Typically  a  MUSTPAC™  system  is  used  as  follows.  First,  the  patient  is  scanned  by  placing  an  ultrasound 
probe  on  the  patient  and  mechanically  sweeping  it  across  his/her  skin  over  the  area  of  interest.  During  the 
scan,  the  system  records  ultrasound  data  from  a  sizable  3D  volume  of  the  patient's  anatomy,  producing  a 
3D  volumetric  data  set  of  ultrasound  reflectivity.  The  scanning  process  requires  no  interpretation  of  the 
ultrasound  images,  other  than  possibly  to  confirm  that  the  intended  anatomy  is  covered. 

Scans  in  the  form  of  3D  volumetric  datasets  are  then  transmitted  over  any  standard  digital  network  to  a 
qualified  diagnostician.  Finally,  a  diagnostician  interprets  each  3-D  scan  using  a  Virtual  Ultrasound  Probe 
that  simulates  a  conventional  real-time  hands-on  examination  procedure.  This  allows  the  diagnostician  to 
display  arbitrary  2D  slices  from  the  3D  data  set  simply  by  moving  the  probe  as  if  he/she  were 
interactively  examining  the  patient.  The  Virtual  Ultrasound  Probe  and  corresponding  screen  displays  are 
very  natural  to  diagnosticians,  leading  to  rapid  acceptance  and  productivity. 

All  versions  of  the  MUSTPAC™  system  developed  to  date  have  used  a  similar  high  level  design.  There 
are  two  primary  operations  —  scanning  and  visualization  —  typically  separated  by  a  data  transmission 
step. 


Mechanical 
Scanning  of 
Transducer 
Probe 
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Video  Capture 


Volume 
Data  File 


1  *'■ 


|  uaia  t-ne 


Visualization  Software 
3-D  Volumetric  +\ 
2-D  Slices 
(arbitrary  reslicing) 


Virtual  Ultrasound  Probe 


Telecommunication: 
file  transfer 

interactive  collaboration 


MUSTPAC-l 

In  the  summer  of  1996,  the  first  prototype  system  was  developed  for  testing  by  the  U.S.  Army  under  field 
conditions.  This  system,  called  the  MUSTPAC-l,  was  designed  to  demonstrate  an  ultrasound 
telemedicine  system  in  a  backpack. 
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System  components  and  packaging.  Major  components  of  the  MUSTPAC-1  are  shown  below.  These 
components  include  (right-to-left): 

•  Battery-powered  linear  scanner  (at  extreme  right) 

•  Silicon  Graphics  Presenter™  flat  panel  display. 

•  Hitachi  EUB-905™  ultrasound  machine  (in  backpack,  top  section,  with  cord) 

•  Silicon  Graphics  Indy™  computer  (in  backpack  bottom  section) 

•  Teleconferencing  camera  (on  backpack,  top  left) 

•  Keyboard  with  integral  touchpad. 

•  High-resolution  color  monitor. 

>  •  Virtual  Ultrasound  Probe 

•  TelelnViVo™  visualization  software. 

•  Other  custom  data  acquisition  and  control  software. 


MUSTPAC-2 

MUSTPAC-2  is  actually  a  family  of  MUSTPAC™  systems  implemented  on  a  common  base  of  Intel 
Pentium  processors  using  the  Windows/NT  operating  system.  The  principles  of  operation  are  similar  to 
MUSTPAC-1.  However,  MUSTPAC-2  incorporates  several  significant  improvements  from  the  user’s 
standpoint.  These  include: 

•  Simplified  and  more  robust  user  interface 

•  Wider  range  of  packaging  options  (e.g.  lightweight  and  rugged  data  acquisition  system  versus 
high  performance  diagnostic  workstation) 

•  Increased  network  compatibility. 

•  Faster  transmission  and  reduced  data  volume  (through  JPEG  data  compression). 

•  DICOM  v3.0  import/export  capability. 
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Freehand  scan  (research  quality-  still  under  development  for  routine  clinical  use). 


Several  configurations  of  MUSTPAC-2  systems  have  been  developed  for  specific  purposes.  These 
purposes  include: 

•  Everest  Extreme  Expedition 

•  Hospital  clinical  evaluation 

•  Remote  clinic  consultation 

For  the  purposes  of  this  proposal,  two  of  the  configurations  will  be  further  developed  for  delivery  to 
Mercy  Health  System  as  MUSTPAC-3.  They  are  the  hospital  and  remote  clinic  versions. 

Hospital  clinical  evaluation.  A  deskside  system  equipped  for  both  scanning  and  visualization  has  been 
installed  at  the  National  Naval  Medical  Center  (NNMC)  in  Bethesda,  MD.  This  system  is  being 
evaluated  for  use  in  a  clinical  setting,  prior  to  possible  shipboard  deployment. 


This  unit  incorporates  a  333  MHz  Dell  Dimension  XPS  computer  (under  desk),  MUSTPAC-2  Virtual 
Ultrasound  Probe  based  on  the  Microscribe  3D  articulated  arm  (on  table),  MUSTPAC-2  linear  scanner 
(on  table),  and  NNMC’s  existing  ATL  HDI-3000  ultrasound  machine  (console  and  cabinet  at  left). 

Remote  clinic  consultation.  The  “lunchbox”  configuration  focuses  on  data  acquisition,  with  limited 
capability  for  visualization.  This  demonstration/evaluation  unit  represents  a  configuration  being 
considered  for  installation  in  a  network  of  telemedicine  clinics  in  rural  northern  U.S. 
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This  portable  “lunchbox”  configuration  incorporates  a  200  MHz  BSI  LCD  V8  computer,  MUSTPAC-2 
linear  scanner  (not  shown),  MUSTPAC-2  Virtual  Ultrasound  Probe,  and  any  of  several  existing 
ultrasound  machines  (also  not  shown). 


5.0  Technical  Approach 

As  outlined  above,  current  MUSTPAC-2  systems  (like  the  one  installed  at  NNMC)  are  based  on  the  use 
of  a  motor-driven  linear  scanning  device,  TelelnViVo™  visualization  software,  and  communication 
protocols  used  at  sites  where  installations  have  been  done  to  date.  These  features  are  not  sufficient  to 
meet  the  requirements  of  Mercy  Hospital’s  intended  usage  of  MUSTPAC. 

To  support  Mercy  Hospital,  Battelle  proposes  to  extend  and  refine  MUSTPAC-2,  thereby  producing  a 
next-generation  system  called  MUSTP AC-3,  and  to  specialize  MUSTPAC-3  for  Mercy  Hospital’s 
operational  environment. 

In  particular,  from  the  standpoint  of  delivered  functionality,  Battelle  proposes  to: 

•  Replace  the  motor-driven  linear  scanning  device  with  a  free-hand  scanning  capability,  where  the 
ultrasound  probe  is  hand-held  and  its  position  and  orientation  are  sensed  instead  of  controlled.  This 
will  allow  the  ultrasound  probe  to  follow  body  contours  and  be  tilted  to  scan  around  obstructions 
(such  as  looking  upward  under  the  rib  cage).  It  will  also  allow  performing  scans  in  many  special 
situations,  such  as  collecting  images  for  cervical  length  measurements  by  using  a  vaginal  probe  with  a 
side-to-side  tilting/sweeping  movement. 

•  Implement  the  free-hand  scanning  capability  using  state-of-the-art  electro-optical  and  mechanical 
technology  that  is  highly  accurate  and  reliable  even  in  the  presence  of  metal,  variable  magnetic  fields, 
etc. 

•  Extend  the  range  of  measurements  provided  by  the  visualization  software  to  include  angles  and 
volumes,  and  to  view  the  original  (non-parallel)  freehand  scan  images,  in  addition  to  the  arbitrary 
reslicing  that  is  currently  provided. 

•  Provide  file  conversion  and  communication  software  as  needed  to  make  MUSTPAC-3  interoperate 
with  Mercy  Hospital’s  existing  system  for  radiology  image  storage  and  retrieval. 
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The  following  actions  are  planned  to  provide  the  above  functionality: 

•  Replace  the  current  visualization  software  (TelelnViVo™)  with  a  different  software  package 
(3D  FreeScan™)  that  integrates  free-hand  scanning  capability  and  visualization. 

•  Utilize  the  MicroScribe™  high  resolution  mechanical  arm  as  a  6-D  spatial  positioning  input  device 
for  both  data  acquisition  (scanning)  and  visualization  (reslicing,  using  the  virtual  probe).  There  are 
two  major  parts  to  this  work: 

•  Extend  the  3D  FreeScan™  software  to  support  the  MicroScribe™  arm. 

•  Design  and  fabricate  mechanical  mounting  devices  to  attach  real  and  virtual  ultrasound  probes  to 
the  MicroScribe™  arm,  and  to  hold  the  arm  in  comfortable  positions  for  clinical  use. 

•  Extend  MUSTPAC™  communication  hardware  and  software  as  needed  to  operate  in  the  Mercy 
Hospital  environment. 

•  Extend  MUSTPAC™  file  conversion  software  for  compatibility  with  Mercy  Hospital’s  image  storage 
and  retrieval  system. 

•  Procure  hardware  and  software  licenses  for  MUSTPAC™  installation  at  Mercy  Hospital  and  a 
development  /  hot  spare  system  at  Battelle,  including  two  each  computer  systems,  MicroScribe™ 
arms,  video  capture  cards,  FreeScan™  licenses,  digital  cameras,  communication  interfaces,  and 
routinely  required  commercial  utility  software  such  as  Microsoft  Office,  disk  defragmenter,  WSFTP, 
PKZIP. 

•  Develop  new  end-user  training  and  documentation  materials  covering  Mercy  Hospital’s  targeted 
applications  for  MUSTPAC™. 

•  Visit  Mercy  Hospital  to  install  MUSTPAC™  equipment,  calibrate  the  MUSTPAC™  system  for  use 
with  the  ultrasound  system  to  be  provided  by  Mercy  Hospital,  and  train  Mercy  personnel  on 
MUSTPAC™  use. 


6.0  Task  Descriptions/Statement  of  Work 

Task  1.  Develop  free-hand  gray  scale  scan  capability  for  the  MUSTPAC  system  that  enables  the 
collection  of  diagnostically  useful  ultrasound  data  by  an  operator  with  limited  training  and  no  ultrasound 
diagnostic  skills,  using  a  simple  “point  and  shoot”  procedure. 

Task  2.  Develop  the  software  to  support:  calibration  of  the  MUSTPAC-3  for  use  with  the  Mercy 
ultrasound  instrument;  interface  of  the  MUSTPAC-3  with  the  Mercy  computer  network,  the  Mercy 
PACS,  and  the  DICOM  standard  in  use. 

Task  3.  Install  free-hand  scan  prototype  on  the  Mercy  computing  platform.  Installation  will  include  a 
free-hand  scanner,  a  diagnostic  virtual  probe  and  the  MUSTPAC-3  software.  The  installation  will  be 
calibrated  to  the  ultrasound  instrument  provided  by  Mercy.  Network  communications  interface,  PACS 
and  DICOM  support  and  data  compression  will  be  confirmed  during  the  installation.  Document  the  setup. 
Provide  user  training  during  the  MUSTPAC-3  installation.  Provide  telephonic  “help”  support  for  90  days 
after  the  installation. 


7.0  Schedule 
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Milestone 

Date 

Task  1.  Complete  MUSTPAC-3  prototype  free-hand  scan 
development 

5  months  after  award 
of  contract 

Task  2.  Complete  software  development  to  support 
MUSTPAC-3  installation  at  Mercy  facility 

4  months  after  award 
of  contract 

Task  3.  Install  MUSTPAC-3  prototype  at  Mercy  facility 

Within  30  days  after 
completion  of  Task  1 

Monthly  Project  Report 

8.0  Budget 

[deleted  from  project  DAMD17-94-C-4127  report] 
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APPENDIX  N 


APPENDIX  N:  “ MUSTPAC ™  3-D  Ultrasound  Telemedicine  Imaging  System”, 
flyer  distributed  at  the  DOE  Biomedical  Technologies  Exposition, 
Washington  DC. 
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